OVERSIGHT HEARING ON THE 
TRAINING PROVIDED TO VBA CLAIMS 
ADJUDICATORS AND THE STANDARDS 
USED TO MEASURE THEIR PROFICIENCY 
AND PERFORMANCE 


HEARING 


BEFORE THE 


COMMITTEE ON VETERANS’ AEEAIRS 
HOUSE OE REPRESENTATIVES 

SUBCOMMITTEE ON DISABILITY ASSISTANCE 
AND MEMORIAL AEEAIRS 

ONE HUNDRED NINTH CONGRESS 

SECOND SESSION 


SEPTEMBER 13, 2006 


Printed for the use of the Committee on Veterans’ Affairs 

Serial No. 109-62 



30-381.PDF 


U.S. GOVERNMENT PRINTING OFFICE 
WASHINGTON : 2007 


For sale by the Superintendent of Documents, U.S. Government Printing Office 
Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC area (202) 512-1800 
Fax: (202) 512-2250 Mail: Stop SSOP, Washington, DC 20402-0001 


AUTHENTICATED 
U.S. GOVERNMENT 
INFORMATION 


COMMITTEE ON VETERANS’ AFFAIRS 

STEVE BUYER, Indiana, Chairman 


MICHAEL BILIRAKIS, Florida 
TERRY EVERETT, Alabama 
CLIFF STEARNS, Florida 
DAN BURTON, Indiana 
JERRY MORAN, KANSAS 
RICHARD H. BAKER, Louisiana 
HENRY E. BROWN, Je., South Carolina 
JEFF MILLER, Florida 
JOHN BOOZMAN, Arkansas 
JEB BRADLEY, New Hampshire 
GINNY BROWN-WAITE, Florida 
MICHAEL R. TURNER, Ohio 
JOHN CAMPBELL, California 
BRIAN P. BILBRAY, California 


LANE EVANS, Illinois, Ranking 
BOB FILNER, California 
LUIS V. GUTIERREZ, Illinois 
CORRINE BROWN, Florida 
VIC SNYDER, Arkansas 
MICHAEL H. MICHAUD, Maine 
STEPHANIE HERSETH, South 
Dakota 

TED STRICKLAND, Ohio 
DARLENE HOOLEY, Oregon 
SILVESTRE REYES, Texas 
SHELLEY BERKLEY, Nevada 
TOM UDALL, New Mexico 
JOHN T. SALAZAR, Colorado 


JAMES M. LARIVIERE, Staff Director 


SUBCOMMITTEE ON DISABILITY ASSISTANCE 
AND MEMORIAL AFFAIRS 

JEFF MILLER, Florida, Chairman SHELLEY BERKLEY, Nevada, 

JERRY MORAN, Kansas Ranking 

JEB BRADLEY, New Hampshire, Vice Chairman TOM UDALL, New Mexico 

GINNY BROWN-WAITE, Florida LANE EVANS, Illinois 

PAIGE MCMANUS, Subcommittee Staff Director 


(II) 


CONTENTS 

September 13, 2006 

Page 

Oversight Hearing on the Training Provided to VBA Claims 
Adjudicators and the Standards Used to Measure their Profi- 
ciency and Performance 1 

OPENING STATEMENTS 

Chairman Miller 1 

Hon. Shelley Berkley, Ranking Democratic Member 2 

Prepared statement of Ms. Berkley 22 

STATEMENT FOR THE RECORD 

Hon. Ginny Brown-Waite 23 

WITNESSES 

Walcoff, Michael, Associate Deputy Under Secretary for Field 
Operations, Veterans Benefits Administration, U.S. Depart- 3 

ment of Veterans Affairs 25 

Prepared statement of Mr. Walcoff 

Wise, Lauress L., Ph.D., President and Chief Executive Officer, 

Human Resources Research Organization 11 

Prepared statement of Dr. Wise 36 

Cox, J. David, R.N., National Secretary-Treasurer, American 

Federation of Government Employees, AFL-CIO 13 

Prepared statement of Mr. Cox 45 

Smithson, Steve, Deputy Director for Claims Services, Veter- 
ans Affairs and Rehabilitation Division, American Legion .. 15 

Prepared statement of Mr. Smithson 56 


(III) 


OVERSIGHT HEARING ON THE TRAINING 
PROVIDED TO VBA CLAIMS ADJUDICATORS AND 
THE STANDARDS USED TO MEASURE THEIR 
PROFICIENCY AND PERFORMANCE 


Wednesday, September 2006 

U.S. House of Representatives, 
Subcommittee on Disability Assistance and 

Memorial Affairs, 
Committee on Veterans’ Affairs, 
Washington, D.C. 

The Subcommittee met, pursuant to call, at 2:30 p.m., in Room 
334, Cannon House Office Building, Hon. Jeff Miller [Chairman 
of the Subcommittee] presiding. 

Present; Representatives Miller, Berkley. Staff Present: Paige 
McManus, Majority Counsel and Mary Ellen McCarthy, Minority 
Counsel. 

Mr. Miller. Good afternoon, everybody. Our Ranking Member 
is here, so we will bring this meeting to order. 

As we all know, the number of veterans filing claims for com- 
pensation has increased every year since 2000, and the claims 
that are being filed are becoming much more complex. As we are 
all very well aware, many veterans are upset with the time that it 
takes to adjudicate a claim for compensation benefits. As of July, 
the most recent data available, it is taking the Veterans Benefits 
Administration an average of 174 days to adjudicate a claim, with 
an accuracy rate of 88 percent. The ability to provide timely and 
accurate benefits is dependent not solely on increasing staffing 
levels, but providing proper and thorough training. 

In recognition of this, the Committee recommended an addi- 
tional 400,000 above the administration’s request in its fiscal 
year 2007 views and estimates to fund training and certification 
initiatives. Congress and certainly VA must ensure that current 
and new employees have the skills and knowledge necessary to 
render accurate and fair decisions the first time. VBA has a num- 
ber of programs and tools that it utilizes to train new Veteran 
Services Representatives, and to retrain or refresh more senior 
VSRs and Rating Veteran Service Representatives. 
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The VBA has implemented a skills certification test to assess the 
knowledge hase of claims adjudicators, and to provide additional 
training when necessary. However, it is my understanding that a 
great many employees either don’t have the time needed to devote to 
training, or are failing basic competency tests. 

We meet today to receive testimony on the types of training provid- 
ed to claims examiners, the standards that are being used to measure 
proficiency and performance, and just what VBA is doing to enhance 
performance. 

I want to take this opportunity to recognize our Ranking Member, 
Ms. Berkley, for comments. 

Ms. Berkley. I thank you very much, Mr. Chairman, for holding 
this hearing, and welcome to all of you. 

VA employees who decide critical benefits issues for our nation’s 
veterans must obviously have proper training in order to fulfill the 
VA’s mission. I am very pleased that we are giving attention to this 
matter today. I am concerned that the majority of experienced VSRs 
who have taken open-book, open-resource tests were not successful 
in passing these tests. I am also concerned that files reviewed by 
Committee staff, and claims brought to the attention of the Commit- 
tee, often contain obvious errors, which VA managers quickly agree 
to correct. 

According to Mr. WalcofPs testimony, the VA provides national 
comprehensive training to regional office staff. However, both the 
employees union, American Federation of Government Employees, 
and the American Legion, report wide regional variations in training. 
Having a national plan does not guarantee that a national plan will 
be effectively implemented. The union testimony concerning VBA 
excluding labor representatives from meetings about training and 
performance standards is, quite frankly, very disturbing. Labor and 
management need to work together to improve the training provided 
to VA employees. 

Veterans who have honorably served our nation deserve quality 
decisions in a timely manner in order to receive the benefits and ser- 
vices they deserve. I am anxious to hear your testimony, both panels, 
and I thank all of you for being here today, and look forward to hear- 
ing what you have to say. 

[The statement of Ms. Berkley appears on p. 22] 

Mr. Miller. Thank you very much, Ms. Berkley. 

Our first panel is already seated at the table. Other members will 
be coming and going as we have other hearings going on this after- 
noon. Testifying on behalf of the Veterans Benefits Administration is 
Mr. Michael Walcoff, Associate Deputy Under Secretary for Field Op- 
erations. He is accompanied by Mr. James Whitson, who is Director 
of the Eastern Area. Ms. Dorothy MacKay is here today, she is the 


3 


Director of the Office of Employee Development and Training, along 
with Ms. Janice Jacobs, Deputy Director of Policy and Procedures at 
the Compensation and Pension Service. 

Mr. Walcoff, you are recognized. 

Me. Walcoff. Thank you, Mr. Chairman, and members of the Sub- 
committee. I appreciate the — 

Me. Millee. If you could pull that mic just a little bit closer. 

Me. Walcoff. Sure. 

STATEMENT OF MICHAEL WALCOFF, ASSOCIATE DEPUTY 
UNDER SECRETARY FOR FIELD OPERATIONS, VETER- 
ANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT 
OF VETERANS AFFAIRS; ACCOMPANIED BY JAMES 
A. WHITSON, DIRECTOR, EASTERN AREA VETERANS 
BENEFITS ADMINISTRATION; DOROTHY MACKAY, DI- 
RECTOR, OFFICE OF EMPLOYEE DEVELOPMENT AND 
TRAINING, VETERANS BENEFITS ADMINISTRATION; 
JANICE JACOBS, DEPUTY DIRECTOR, POLICY AND PRO- 
CEDURES, COMPENSATION AND PENSION SERVICE 

Me. Walcoff. I appreciate the opportunity to appear before you to- 
day to address training for Veterans Benefits Administration claims 
adjudicators, the standards we use to measure their proficiency and 
performance, and how we communicate changes in laws, regulations, 
policy, and procedures, to field station staff. 

I am being accompanied by my VBA colleagues; Mr. Jim Whitson, 
who is the director of our eastern area office and also the Project 
manager for C&P certification; Ms. Janice Jacobs, deputy director of 
compensation and pension service; and Dorothy MacKay, director of 
our office of employee development and training. 

Training is essential to every quality organization. VBA is com- 
mitted to ensuring all employees have the opportunity to learn and 
develop essential knowledge, skills, and abilities. We recognize effec- 
tive training as a core element of VBA’s infrastructure, and key to our 
succession planning efforts, as we hire to replace the many experi- 
enced employees expected to retire over the next few years. VBA has 
deployed new training tools and programs to provide consistency in 
training. Newly hired veteran service representatives, or VSRs, and 
newly appointed rating VSRs participate in consistent, nationally- 
developed centralized training, followed by a national standardized 
training curriculum at their home offices. 

This fiscal year, the undersecretary for benefits. Admiral Daniel 
L. Cooper, mandated all claims examiners to have at least 70 hours 
of job-specific training. Most other employees must have at least 40 
hours of training. In fiscal year 2007, the mandatory training for 
claims adjudicators will increase to 80 hours. 
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VBA ensures regional offices have timely, accurate, and current in- 
formation. C&P service issues guidance letters, called “fast letters,” 
to advise field employees on policy and procedural changes, and legal 
revisions. C&P service issued 23 fast letters in 2005, and 16 to date 
in 2006. 

Decision assessment documents, or DADs, explain the facts, rea- 
soning, and holding of precedential court decisions, and the opinions 
of the office of General Counsel, and explain any impact on VBA. 
Notification of fast letters, DADs, and other changes, are e-mailed to 
field employees and posted on the C&P service’s wehsite. 

C&P service uses satellite broadcasts to give in-depth analysis on 
VBA procedures and policies hy subject matter experts, and hosts 
monthly conference calls to discuss current hot topics and give in- 
stant feedback on questions and concerns. 

Regarding VBA, VSR, and RVSR national performance standards, 
a VBA organizational cornerstone to improve benefit delivery and en- 
hance accountability is our system of individual performance assess- 
ment. All VSRs and RVSRs are subject to national performance stan- 
dards that focus on key elements of quality, productivity, customer 
service, and timeliness. Local accuracy reviews are conducted using 
national quality review criteria. Agreements were reached with the 
American Federation of Government Employees prior to implemen- 
tation. The current VSR performance standards have been in effect 
since October of 2005. The RVSR performance standards have been 
in effect since November of 2001. The VBA work group is reviewing 
these standards for possible revision. 

Regarding certification testing; in conjunction with our national 
performance standards, VBA developed a skill certification testing 
process to assess job proficiency. A memorandum of understanding 
was signed in January 2000 with AFGE for a certification program 
for VSRs, RVSRs and decision review officer positions. 

In December 2000, VBA contracted with the research group, the 
Human Resources Research Organization, or HUMRO, to help de- 
velop a certification instrument with VBA subject matter experts to 
assess knowledge and readiness of VSR GS 10 incumbents for promo- 
tion to the GS 11 level. 

A 100 question multiple choice open-book test is administered in 
morning and afternoon sessions. In August of 2003, 25 percent of the 
298 participants passed the first validation test. In April of 2004, 29 
percent of the 650 participants passed the test. After the first two 
tests, a mandatory 20-hour VSR readiness training curriculum was 
implemented. 

On May 3rd, 2006, 934 VSRs took the third test. During the after- 
noon portion, multiple test items were duplicated from the morning 
portion. The correct version of the test was given on June 7th, and 
the third test yielded an improved pass rate of 42 percent. To date. 
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VBA has promoted 633 VSRs to the full-performance GS 11 level, 
through certification testing. We are expanding certification testing 
next, to the RVSR position, from the GS 11 level to the GS 12 journey- 
man level. 

Mr. Chairman, this concludes my testimony. I greatly appreciate 
being here today, and look forward to answering your questions. 

[The statement of Michael Walcoff appears on p. 25] 

Me. Miller. Thank you very much for your testimony. I think we 
hoth have some questions, and one of the issues I think that we prob- 
ably both want to know about is when a case is reprimanded from 
the Board for further development, it is generally sent to the Appeals 
Management Center in Washington, DC. Is the employee who made 
the initial decision - first question - is the employee ever notified of 
what the deficiencies were, or what may have happened in the ap- 
peal? You know, I would think that if they are not, then it probably 
would help the R.O. directors develop a better understanding of how 
ratings happen, and where the issues are. Can you go into that just 
a little bit? 

Mr. Walcoff. Sure. Let me first say that when a case is remanded 
from the board, it is very often three, sometimes four years after it 
was originally worked, for a lot of different reasons. Very often, the 
case that is remanded back to VBA doesn’t look like it did when it 
left the regional office. Additional information has been sent in by 
the veteran. Many, many cases are requiring a new exam, and that 
is a reason why a case might be remanded. Sometimes, the rating 
schedule may change. There might be a change in regulation that 
might cause for the case to be remanded. And certainly, in a small 
percentage of cases, a percentage of cases, it is similar to when the 
VSR finished it, or the rating specialist finished it, and it is because 
of an error that was made at that time. But it is a combination of all 
those factors that go into the remand rate. 

When we originally set up the AMC, our discussions with the prior 
secretary dealt with how many of the remands that came out of the 
board should be done at the AMC, and how many should be returned 
to regional offices. It was Secretary Principi’s opinion that he wanted 
all the remands to be processed at the AMC. He felt that this was the 
most efficient way to work through the remands. He was concerned 
about the fact that there were a lot of older remands that were still 
out there, and he felt that this would be the best way to deal with 
that. 

We had some very serious discussions with him about why we felt 
that, at minimum, a certain percentage of these cases needed to go 
back to regional offices. One of the reasons, quite frankly, was that 
in people working these cases, and they have the sense that “no mat- 
ter what I put on this piece of paper, I will never see the case again,” 
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that there is almost a little hit of a disincentive in terms of being 
much more careful about what they are sending to the board, and 
making sure that it is ready for the case to be adjudicated by BVA. 
And we were concerned about that. We wanted to make it so that 
some of these cases do go back to regional offices. 

But more importantly, we felt that it was important that some of 
these cases go back to R.O.’s for training purposes. It is very impor- 
tant for people who are working on certifying cases to the board, that 
they get to see what types of cases are being remanded, so they can 
use that information in doing the work that they are currently doing. 
So it doesn’t necessary to go back to the original person that prepared 
the case, because as I say, it is many years later. 

But they do see a good number of the cases. And they particularly 
see what we call the egregious cases, the ones where there is really 
absolutely no excuse for why this case ever left the regional office. 
And we want to make sure that those cases are reviewed by the re- 
gional offices, and by the management at those offices to make sure 
that it doesn’t happen again. 

Me. Miller. In its 2005 report on state variances in compensa- 
tion payments, the VA Inspector General surveyed about a thousand 
employees on factors that affect consistency of rating decisions. The 
results showed that while 60 percent of the employees had positive 
opinions about the quality of the training that they were receiving, 
many feel that the training is not a high priority, and some 37 percent 
or so indicated that formal classroom training occurs once a quarter, 
or less often than that. Can you talk to us a little bit about what is 
being done? 

Me. Walcoff. I would say that the Under Secretary, Admiral Coo- 
per, has I think probably stressed training, training and consistency, 
probably more than any other subject since I have been working with 
him. I have been in Washington since 1998, and I have served under 
several under secretaries, and was here previously, back in the early 
‘90s with two other under secretaries. I have never worked for any- 
body that was, I believe, as serious about training as Admiral Cooper 
is, and I think it is because of his background with all the years he 
spent in the Navy. 

I will tell you that it is one thing to say that training is important, 
and “here is a mandatory requirement that you do 70 hours.” But the 
real question is, how do you ensure that is actually happening? And 
I will tell you a couple different ways that I think we are going about 
doing that. One is I think we are providing very close oversight when 
we go to regional offices, to make sure that training is being done and 
is being done effectively. The C&P service review goes to about 18 of- 
fices a year, and to review the C&P operations at each office. And one 
of the main things that they look at is the training that is being done; 
whether the curriculum that is specified by the C&P service is being 
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followed, and whether all employees are going to the training. 

In addition to that, our area directors visit their offices two, three, 
four times a year. And one of the things that they look at is the same 
thing: how is the training being done? And they will not just talk to 
the managers. They will actually go and talk to the VSRs, and talk to 
the rating specialists, and ask them, you know, “How much training 
have you received,” you know, “Did you actually attend this class? 
Were you at the satellite broadcast?” Those types of things, to make 
sure that it is happening. 

Secondly, we have a log that has been put together by Dorothy’s 
staff that requires the managers to report every hour of training that 
is given to employees, so that we can verify that 70 hours of training 
this year, in this case, was actually given to every employee in re- 
gional offices in the service center. So we also review the logs. 

And the third thing that we do, and this is something that I haven’t 
seen in all the years I have been working, is that Admiral Cooper 
personally reviews the training report that is done twice a year by 
each regional office, that goes into great detail about all the training 
that they have done during that previous six months. And this is not 
something that comes in and he has Dorothy give him a summary re- 
port. He actually has to see the reports. He reads every one of them, 
marks them all up with comments, and sends them out back to her 
and to me, you know, with questions, and wanting us to follow up on 
things, obviously showing that he is sincerely interested in it. 

So I think that aside from having the requirement, there is a lot of 
follow-up to make sure that it is actually being done. This is a new 
requirement. I mean, this is the first year that we actually had that 
mandatory 70 hours. And I think it was a little bit of a shock to our 
organization, frankly, that somebody did more than just say, “Yeah, 
I think training is important,” but actually put out a mandatory re- 
quirement, you know, saying that “You will do training.” That has 
had an impact on the organization. 

Me. Miller. Ms. Berkley? 

Ms. Berkley. Thank you, Mr. Chairman. In reviewing panel two’s 
testimony, a couple of things struck me, and I would like to ask you 
questions based on the testimony that was submitted by the other 
panel. 

According to AFGE’s written testimony, it indicated that unreal- 
istic performance standards and inconsistent training adversely im- 
pact the quality of employees’ work. They report also being excluded 
from the task force meetings where current training and performance 
standards are discussed. Can you share with us why we would ex- 
clude these key stakeholders, and why they are not at the table when 
discussions of this magnitude are being — when they are discussing 
these issues? 

Mr. Walcoff. Well, first of all, I would tell you that I think we 
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have a very positive relationship with AFGE. The speaker that will 
he representing AFGE here in the second panel is somebody that I 
have worked personally with for many years, and I think that while 
we may disagree on individual issues, I think he would agree that the 
VBA and AFGE have had a good relationship. 

I think that we certainly accept that there is an appropriate role for 
labor, for AFGE. We certainly understand all the things that we are 
responsible to do in terms of our relationship with them, and I think 
that we very clearly meet every requirement that we have in terms 
of our relationship. 

We do include them on many of these types of issues that we work 
on. The RVSR performance standards, for instance, that are in place 
right now, the union had a representative on that team. They partici- 
pated in setting up those standards. That is aside from the fact that 
we then negotiated an agreement with them about the standards that 
are in question right now. The VSR standards, which is I think what 
David is referring to in his statement, we negotiated an agreement 
with them on the implementation of those standards. 

And the disagreement that we have right now with them that he 
has mentioned in his testimony has to do with a requirement in the 
MOU for us to review the actual performance under those standards, 
and make changes, and there is a difference of opinion as to whether 
we have done that or not. But we certainly have worked with the 
union on the implementation of those standards, and met every re- 
quirement that is there for our labor-management relationship. 

In terms of the standards being too difficult, I respectfully disagree. 
I think that some of our managers feel that we should have set the 
floors higher. We make it clear that it is not just a productivity re- 
quirement; that there is an absolute quality requirements in there, 
as well as a timeliness and a customer-service requirement. They are 
all considered critical. And if you look at the performance of the field 
in those standards, right now we have a pass rate of over — in terms 
of meeting all the requirements of their position — of over 80 percent 
within the VSR position. And that goes up every month, as people 
become more familiar with the changes that we made almost a year 
ago in the way we measure those standards. 

Some of the problems that we have had had to do with the track- 
ing system that is designed to go with those standards, called As- 
pen. There has been — it is a new system, and it is taking quite a 
while for some of our employees, as well as managers, to understand 
how to use it. And as we reviewed the pass rates in preparation for 
the grievance that is mentioned in David’s testimony, we found that 
there were some R.O.’s that weren’t using the tracking system cor- 
rectly, and that was why some of the pass rates look lower than they 
actually are. But we are very confident that the pass rate is over 80 
percent, and going up every month. 
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Ms. Berkley. Do you think there is inconsistent training? And 
does it adversely impact on the quality of the employees’ work? 

Mr. Walcoff. You know, inconsistent training is something that 
we are always very concerned about, and we have done everything, I 
think, that we can to try to deal with that. That is why we have not 
only — not only do we have a national curriculum, starting with the 
basic challenge training for all of our new employees, where they go 
to a centralized place and spend three weeks there, and then when 
they go back to their regional offices, spend 23 weeks taking a na- 
tional TPSS curriculum that everybody is required to take. 

But for journeyman VSRs and rating specialists, we get the 70- 
hour requirement, where C&P service has listed a number of courses 
that are mandatory for every individual during that year, and also 
specifies “This is the way you will teach it. Here is a satellite broad- 
cast,” you know, “Here is a video,” that type of a thing. So certainly, 
mechanisms are out there to ensure that there is consistent training. 
We track the logs to make sure that everybody is doing what they are 
supposed to do consistently. You know, in an individual — 

Ms. Berkley. There is adequate oversight, in your opinion? 

Mr. Walcoff. I believe there is. I believe there is, ma’am. 

Ms. Berkley. Let me draw your attention to the American Legion 
testimony. In it, they have indicated that the focus on end-product 
production quotas in regional offices leads to poor-quality decisions 
and appeals. When and how did the VA last evaluate the usefulness 
of end-product code measure of performance? And is there perhaps a 
better method of measurement? 

Mr. Walcoff. I think that everybody, or just about everybody, 
would agree that we would like to evolve away from the basic end- 
product system, to more of an issues-oriented system, and I have 
been to hearings where this has been discussed. And I think you 
know that up until about a year ago, we really didn’t have the abil- 
ity to accurately measure the work that we are doing or receiving on 
an issues-based basis. With RBA 2000 now being mandatory, and 
everybody using it, that is really a vehicle that really does allow us to 
be able to count issues versus cases. 

And that is really the first big step toward moving away from an 
end-product system. It is certainly a more accurate way to measure 
quality. It is a more accurate way to measure our production. And it 
would do a lot in terms of some of the concerns even that AFGE has 
about how we measure how productive, you know, an individual is. 
We have talked a lot about it. 

There are some initial steps taking place for us to begin doing that. 
It is going to be a process. I mean, it is certainly something that were 
going to have to work with, with all of you all on, in terms of explain- 
ing what we are doing, why we are doing it, and how that will affect 
our performance, because we want to be able to give you apples and 
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apples to be able to compare in terms of whether we are improving or 
not. Certainly we would work with service organizations, and with 
the union, as we migrate from the one system to another. It is not 
something that is going to be done over a period of months. But we 
have had discussions with Admiral Cooper, and I know that he is 
committed that this is something that were going to want to do. 

Ms. Berkley. Is there anything that we can do to help implement 
that, in getting away from the end-product system? Because I agree 
with you; it won’t take months. But how do we, as members of this 
Committee, enable you to do this and not be here 20 years from now 
talking about the same thing? 

Me. Walcoff. One of the things that I think will be helpful is, you 
know, any time we make a major change in the way we measure 
something, you know, I think that there is always a concern — and 
as I say, I have been around in headquarters for a while, and I know 
that sometimes there is a concern, “Well, they are kind of changing 
the name of the game a little bit to maybe just take away attention 
from what actual performance is, and that sort of thing.” And I think 
certainly that is a legitimate concern that our stakeholders certainly 
should be thinking about. 

What I would want to do in terms of working with you is to try to sit 
down and talk about, “Okay, what is it that you would need for reas- 
surance as we evolve from one system to another?” So that you could 
see not only that we are moving, you know, in a direction that we both 
think we need to, but you can also monitor on us in your oversight 
capacity as to how we are performing, and be able to do it on some 
consistent plane, rather than in two different languages, sort of, you 
know, so to speak. 

And that is something we will be able to hopefully work with you 
on, and you know, I think we can come to some agreement on that. 

Ms. Berkley. Okay. Thank you, Mr. Walcoff. 

Me. Miller. Thank you very much. We appreciate your testimony. 
Thank you for those that joined you today. If you would, if you could 
have a member, you or a member of your staff, if they could remain 
behind we would appreciate it, for the next panel. Thank you very 
much. 

Me. Walcoff. Okay, thank you. 

Me. Miller. And if the second panel will make their way forward. 
While you are getting settled I am going to go ahead and introduce 
everybody if it is okay. 

Dr. Lauress Wise is president of the Human Resources Research 
Organization, the group VBA contracted with to develop the skills 
certification testing program. Mr. Jeffrey David Cox is the national 
secretary-treasurer of the American Federation of Government Em- 
ployees. And Mr. Steve Smithson, he is with us, is the deputy di- 
rector for claims services at the Veterans Affairs and Rehabilitation 
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Division of The American Legion. 

And each of you we would ask, if you would, hold your testimony 
to five minutes, since there are three of you. Your full testimony will 
he entered into the record. It will he printed in its entirety, as it was 
submitted. 

Dr. Wise, welcome, and you may begin. 

STATEMENTS OF LAURESS L. WISE, PH.D., PRESI- 
DENT AND CHIEF EXECUTIVE OFFICER, HUMAN RE- 
SOURCES RESEARCH ORGANIZATION; J. DAVID COX, 
R.N., NATIONAL SECRETARY-TREASURER, AMERI- 
CAN FEDERATION OF GOVERNMENT EMPLOYEES, 
AFL-CIO; STEVE SMITHSON, DEPUTY DIRECTOR FOR 
CLAIMS SERVICES, VETERANS AFFAIRS AND REHABILI- 
TATION DIVISION, AMERICAN LEGION 

STATEMENT OF DR. LAURESS WISE 

Dr. Wise. Thank you. Good afternoon, Mr. Chairman. I am Lau- 
ress Wise, president of the Human Resources Research Organization, 
known less formally as HUMRO. HUMRO is a nonprofit 501(c)3 re- 
search and development organization established in 1951 that worked 
with government agencies and other organizations to improve their 
effectiveness in developing and managing their human resources. 

I have been asked to testify today about the work that HUMRO 
has done for the Veterans Benefits Administration on their program 
for certifying essential skills for veteran service representatives. For 
brevity, veteran service representatives are referred to as VSRs. 
These service representatives play a key role in seeing that our veter- 
ans receive a full array of benefits to which they are entitled. Perfor- 
mance at the highest level of this position requires a thorough under- 
standing of an extensive set of policies and procedures, and skill in 
identifying appropriate applications of these procedures to individual 
circumstances. 

The skill certification program embarked on by the VBA is criti- 
cal to ensuring that service representatives have the knowledge and 
abilities needed to perform their jobs effectively. 

My written testimony, which I ask be entered into the record, de- 
scribes in some detail HUMRO’s efforts to develop the VSR skill cer- 
tification test. I will just cover briefly the essential points here. Our 
work began in 2001 with an analysis of critical tasks performed by 
the VSRs and the knowledge and skills required to perform these 
tasks effectively. HUMRO worked with a design team that included 
VBA management, expert job incumbents, union representatives, 
and members of the veteran service organizations, to review the job 
analysis results, and develop a blueprint for the certification test. 
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The blueprint identified content areas to be covered by the test and 
specified the number of test questions to be included for each of these 
content areas. We worked with the design team to develop, review, 
and provide high-quality test questions. A pilot test was conducted 
to screen individual questions followed by a field test of the entire 
certification process. 

Two different approaches were taken to accessing the validity of 
the resulting test scores for certifying a candidate’s qualification for 
the GS 11 position. In a content validity study, experts affirmed that 
each of the test questions measured knowledge and skills essential 
for job performance. We also conducted a criterion-related validity 
study. In that study, test scores were found to be significantly corre- 
lated with job performance ratings made by each examinee’s supervi- 
sors. The test development and validation process conformed fully to 
professional standards for test development and use. The resulting 
certification test is a valid and important tool for ensuring essential 
skills for those promoted to the highest level VSR job. 

I was also asked to comment on the issues surrounding the May 
3rd administration of the skill certification test. Two forms of the test 
were administered to 934 candidates. The reason for administering 
two forms was so that examinees sitting next to each other would not 
get the same questions in the same order. Each test form included 
100 scored items and 20 additional questions being pilot tested for 
future use. The two forms had 67 operational, or scored, items in 
common, albeit in different locations within each test form. 

Because of the length of the test, the test is split across two ses- 
sions; one in the morning and one in the afternoon. Candidates re- 
ceived a different test booklet for each of these two sessions. Due to a 
processing error at HUMRO, some of the questions for the afternoon 
booklets were inserted into the incorrect test form. This error re- 
sulted in duplicating some questions from the morning session in the 
afternoon session booklets for that corresponding test form. Quality 
control procedures in effect at the time included a review of each test 
but did not include a thorough comparison of the morning and after- 
noon booklets within a test form. Consequently, this processing error 
was not caught prior to the test administration. 

The week following the May administration, HUMRO Vice Presi- 
dent Beverly Dugan, our project director. Dr. Patricia Keenan, and I 
met with VBA leadership to discuss the problem and identify meth- 
ods for providing valid scores to the May administration examinees. 
Our discussion identified several possibilities, including using some 
of the pilot test items to construct an 80-item test; ignoring the re- 
dundancy and scoring each of the duplicate items to provide a 100- 
item test; and conducting a supplemental administration, using the 
items that were originally intended to be included in each of the af- 
ternoon booklets. 
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The solution selected by the VBA management was to conduct a 
supplemental test, and administer the items that were originally in- 
tended to be presented in each of the afternoon tests. This allowed 
everyone to be scored on 100 separate items, to keep the test mapped 
to the blueprint exactly as designed, and made the May 2006 ad- 
ministration much more equivalent to the operational field test and 
validity test, and to those administrations planned for the future. 

The supplemental test administration was held on June 7th, 2006. 
After both portions of the test were scored, 42 percent of those who 
took the test passed. The supplemental testing did create an incon- 
venience to examinees, and additional burden to those who adminis- 
tered the test, but the end result was an assessment that covered the 
content framework as intended, with questions and scores that were 
psychometrically sound. 

HUMRO’s staff have reviewed the factors that led to the error in 
assembling the test booklets. One such factor was the limited time 
available for assembling and checking booklets. The VSR job contin- 
ues to evolve. New types of cases are often added to the case load. 
Newly electronic tools and databases are developed, and more pen- 
sion cases are being moved to pension maintenance centers. 

A workshop to review test questions, to be sure that they reflected 
current policy and processes, was held in April of 2006. Item writers 
reviewed all of the items in the bank, revised many of them, and up- 
dated the references. The revisions were more extensive than origi- 
nally anticipated, and the work to update the item bank was com- 
pleted late the week prior. We had only two days to select the forms. 
In retrospect, it was clear that more time is needed. 

We have now implemented procedures to prevent future occur- 
rence of these problems, both by allowing more time, and incorporat- 
ing more thorough checks of the test booklets. 

In summary, the VSR test is an important tool for improving the 
effectiveness of the VSR workshop, and serving the benefit needs of 
our veterans. 

Mr. Chairman, that concludes my testimony. Thank you for the 
opportunity to appear today. 

[The statement of Lauress Wise appears on p. 36] 

Me. Miller. Doctor, thank you very much for your testimony. What 
I would like to do if we could before we go to questions is let the other 
panelists go ahead with their remarks. 

Mr. Cox, you are recognized. 

STATEMENT OF MR. J. DAVID COX 

Me. Cox. Thank you, Mr. Chairman. 

Chairman and members of the Subcommittee, good afternoon. I 
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am J. David Cox, national secretary-treasurer of the American Fed- 
eration of Government Employees. Thank you for inviting AFGE to 
testify today. I ask that my written statement he submitted for the 
record. 

Me. Miller. Without objection. 

Me. Cox. As the purse Vice President of AFGE’s national VA coun- 
cil and a local president, I have visited a large number of VBA re- 
gional offices. What I observed from the front lines is that VSRs and 
rating specialists are under tremendous pressure to process enough 
cases to meet performance standards. They stay late into the evening 
and work weekends. They study on their own time because training 
is often cut short to make the numbers. 

Despite all this, many of them cannot make the numbers. They 
are further demoralized by a confusing certification test. VSRs and 
rating specialists want to help veterans get an accurate and prompt 
decision on their claims. Many are themselves service-connected vet- 
erans. VBA training is clearly an investment worth making. AFGE 
believes that genuine collaboration on training and performance 
measures will make this critical investment go far. Lawmakers, 
managers, employee representatives, and veterans groups, have to 
work together to come up with training and standards that are work- 
able, doable, and durable. 

Unfortunately, VBA seems less and less open to collaboration than 
in the past. After we have worked together to develop the CPI model, 
we have now been left out of key task force meetings. After we worked 
together on national performance standards, they have refused to ad- 
dress problems that have been developed. Even though we developed 
the VSR certification by agreement, VBA is refusing to work with us 
to fix what isn’t working. The role of on-the-job training for VSRs and 
rating specialists cannot be underestimated. 

By VBA’s own estimates, it takes at least two to three years for a 
new VSR to be fully productive. Yet, what VBA is doing about an im- 
pending shortage of experienced adjudicators due to retirement and 
high turnover among frustrated new employees. 

AFGE offers the following recommendations: a joint VBA stake- 
holder team should develop a national mandatory training plan, with 
strong oversight. We need a uniform curriculum, training schedule, 
and a set of best practices. The oversight process should be transpar- 
ent to all stakeholders, with regular reports to Congress to make sure 
that VSRs and rating specialists everywhere get the same quality 
and amount of training, and rotate through all adjudication teams. 

VBA should develop a cadre of trainers skilled in both teaching 
and veterans benefits. Currently, trainers are selected without con- 
sistent criteria. All trainers should be of the caliber one is likely to 
find at the VBA Academy. Continuing education should be manda- 
tory and nationally uniform. Supervisors should be required to hold 
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weekly meetings to go over new laws, regulations, court cases, and 
best practices. 

Current performance standards need revision. One third of VSRs 
and rating specialists are failing to meet unrealistic production quo- 
tas. CPI specialization should also be factored in. Training for skills 
and certification tests should be nationally uniform. The first two 
validity tests for VSR certifications had a very low passage rate, and 
more than half failed the May 2006 test. I can’t help but wonder 
whether any managers have ever had to take this test, and what 
their pass rate would be. 

The test is supposed to be a culmination of the training, but there 
is a real disconnect between the two. Trainers do not even know what 
to cover. Employees are not told how many questions they need to get 
right, which questions they got wrong, and what the correct answer 
is. Something is wrong when two employees with identical scores test 
at different times, and only one passes. 

Finally, we hope the Subcommittee will consider the related issue 
of classification of the VSR and rating specialist positions. Unlike 
comparable adjudication jobs at other Federal agencies, the VSR ca- 
reer ladder ends at the GS 10. We currently understand that VBA 
recently completed a review of these classifications. Once again, we 
would like to be part of this process. We look forward to working 
with Chairman Miller and Ranking Member Berkeley, to ensure that 
VBA claimants get the best services possible. Our veterans deserve 
nothing less. 

Thank you, Mr. Chairman, for allowing me to be here today. 

[The statement of J. David Cox appears on p. 45] 

Me. Miller. Thank you, Mr. Cox. Mr. Smithson, you are recog- 
nized. 


STATEMENT OF MR. STEVE SMITHSON 

Mr. Smithson. Mr. Chairman and members of the Subcommittee, 
thank you for the opportunity to present the American Legion’s views 
on the training provided to Veterans Benefits Administration claims 
adjudicators and the standards used to measure their proficiency and 
performance. We commend the Subcommittee for holding this hear- 
ing to discuss these important issues. I am going to limit my remarks 
to emphasize major points discussed in my written testimony, and 
ask that my written statement be entered into the record. 

Me. Miller. Without objection. 

Me. Smithson. The American Legion appreciates the importance 
the Undersecretary for Benefits has placed on training, and the im- 
provements that have taken place, including centralized training for 
regional office staff conducted by the compensation and pension ser- 
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vice. However, in our opinion, the culture of VA’s claims adjudica- 
tion has heen and continues to he production-driven. This results in 
everything else, including quality assurance and training, taking a 
hack seat. If this culture is ever going to change, VA, in addition to 
the centralized training currently taking place, must consistently im- 
plement national training requirement or standard in each regional 
office for all employees, new hires and experienced alike. 

It must also he implemented with strong oversight directly from 
the Under Secretary for Benefits’ office, and it must have teeth. Man- 
agement stations not in compliance must he held accountable, or the 
effort will not he successful. 

Additionally, there must he an emphasis placed on using informa- 
tion obtained from BVA decisions, DRO decisions, and errors noted in 
the National Star reviews, and other internal quality reviews. Such 
data should be tracked, examined for patterns, and used in formal, 
customized training at each R.O. 

Although such data is being collected and the American Legion’s 
quality review visits at VA regional offices have identified some sta- 
tions that have been conducting such training, it needs to be done 
across the board at each R.O. Unless regional office staff, both man- 
agers and individual adjudicators, learn from their mistakes and take 
corrective action, there will continue to be a high rate of improperly 
adjudicated claims, resulting in consistently high appeals rate, and a 
subsequent high VBA remand and reversal rate. 

Performance standards, which tend to go hand-in-hand with the 
production-first mentality entrenched in VBA is another topic that 
needs to be addressed. Performance standards of claim developers 
and raters are centered on productivity as measured by work credits, 
known as end products. Both veteran service representatives and 
rating service representatives have minimum national productivity 
requirements that must be met each day. Unfortunately, the end 
product work measurement system, as managed by the VA, does not 
encourage regional office managers to ensure that adjudicators do 
the right thing for the veteran the first time. For example, denying a 
claim three or four times in the course of a year before granting ben- 
efits sought allows for several end-product work credits to be counted 
just for this one case, rather than promptly granting the benefit and 
taking only one work credit. 

In the view of the American Legion, the need for substantial change 
in VBA’s work measurement system is long overdue. A more accu- 
rate, reliable work measurement system would help to ensure better 
service to veterans. Ultimately, this would require the establishment 
of a work measurement system that does not allow work credit to be 
taken until the decision in the claim becomes final; meaning that no 
further action is permitted by statute, whether because the claimant 
has filed to initiate a timely appeal, or because BVA rendered a final 
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decision. 

Lastly, we are pleased that C&P has begun implementing a joh 
certification test for VSRs, and we look forward to similar testing 
being established for RVSRs and DROs. However, we are concerned 
that the current testing taking place is not required as a condition 
of employment for the position. Although successful completion of 
the test is required for promotion or transfer to the rating board, it 
is still optional. The ultimate goal of a proficiency or competency 
testing should be to ensure that an individual in any given position 
is competent, proficient, and otherwise qualified to perform the du- 
ties required of that position. This goal will not be achieved if testing 
is not mandatory and remedial training or other corrective action is 
not required for those who do not successfully pass the test. Although 
this concept may not be embraced by some, the ultimate goal is to 
have qualified and competent staff who will provide the best possible 
service for our veterans. 

In closing, the American Legion realizes that VBA faces many diffi- 
cult challenges during the upcoming fiscal year. Although we have of- 
fered our suggestions and comments, we realize that there is no easy 
solution, and we will continue to work closely with VA to ensure that 
our nation’s veterans receive the benefits to which they are entitled. 

That concludes my testimony and I will be happy to answer any 
questions. 

[The statement of Steve Smithson appears on p. 56] 

Me. Miller. Thank you very much, and I will start with you. You 
talked in your written testimony about too few experienced supervi- 
sors were available to provide trainees, I guess, with proper mentor- 
ing and quality assurance. Can you give me some type of an indica- 
tion as to what you would think an experienced supervisor would 
be? 

Mr. Smithson. An experienced supervisor would be somebody that 
has been on the job for a number of years, has worked up through the 
system, and has proven themselves to be competent through their 
work, the performance standards, and other measures. 

Mr. Miller. Mr. Cox, you had talked about VA failing the VSRs 
by not adequately preparing them to take the skills certification test. 
Did I read that correctly? How much of the onus of passing a test 
falls on the employee? 

Mr. Cox. There would be a part that would fall on the employee, 
certainly, sir. However, much of the training material for the train- 
ing for the test, and for the employee to study, is on the computer, sir. 
And I wouldn’t even raise the issue of an employee taking a computer 
home at this point to study for the test. But I mean those type of 
things are the things readily available. And at the same token, this 
is not a job that an employee comes with a ready set of skills. I am a 
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registered nurse by profession. I come to the VA with a set of skills 
required for that job. With a VSR, that person comes to the VA, and 
it is on-the-job training, and it is all done there on the job. 

Me. Miller. Would AFGE support mandatory testing as a condi- 
tion of employment? 

Me. Cox. I think AFGE would have to look very, very closely, and 
be concerned with the mandatory testing as a condition of employ- 
ment, sir. We don’t believe that is done at other Federal agencies 
with similar type jobs, similar to the claimant jobs; Social Security 
being one of them. 

Mr. Miller. Dr. Wise, you indicated that some of the test ques- 
tions in the pilot test were dropped because participants indicated 
that there were problems with those questions. Can you give us an 
idea of what those might have been? 

Dr. Wise. Well, there certainly are instances where the rules may 
have evolved since the question was written, so that no longer clear 
which is the correct option on the question, would be an example. 

Me. Miller. Ms. Berkley? 

Ms. Berkley. Thank you, Mr. Chairman. The State of Nevada has 
the fourth highest remand rate in the nation, and I have a lot of very 
unhappy veterans, and I have been trying for quite a while to figure 
out what we could do to make this work a little bit better. I have 
questions for each of you, and again, let me thank you all for being 
here and sharing your knowledge with us. 

Mr. Smithson, in your experience, do regional offices which provide 
consistent, high-quality training have fewer errors and lower remand 
rates? 

Me. Smithson. Could you repeat the question please? 

Ms. Berkley. Yes, I can. Do regional offices - this seems like a no- 
brainer to me, but I am kind of curious, regarding your experience. 
Do regional offices that provide consistent, high-quality training have 
fewer errors and lower remand rates? Have we been able to track 
high-quality training, better service on the job, lower remand rates? 

Me. Smithson. In our experience, there is more satisfaction. Vet- 
erans have more satisfaction from receiving decisions from those of- 
fices, yes. 

Ms. Berkley. I listened to your testimony, but I have to tell you I 
think you speak faster than I hear. So I would appreciate if perhaps 
you spoke a little slower. Do you have any specific recommendations 
for revision of VBA’s workload management system? 

Me. Smithson. Again, I think we need to — are you talking about 
the end-product, the credit, work credit? 

Ms. Berkley. That could be part of it. Sure. 

Me. Smithson. For the work credit, I think we need to look at not 
giving credit for a case until it becomes final, whether that be the one- 
year period expires and the veteran does not file a timely appeal, or 
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the BVA renders a final decision. 

Ms. Berkley. When we talk about VBA’s workload management 
system, is there a magic number that — and I know it can’t be the 
same number for every employee; everybody works differently. But 
is there a point at which our VBA employeess are so overloaded that 
they can’t possibly provide the quality service that our veterans 
need? 

Mr. Smithson. I think some stations, they are definitely over- 
worked, not enough staff. When we do our quality review visits, not 
only do we look at the decisions for quality errors; we talk to the VBA 
staff, and a lot of them tell us that, you know, there is not enough 
people, the standards are too high, they are not realistic. Some sta- 
tions, we encountered in our experience, in addition to the national 
standard have their own standards that may be above the production 
standards, that may be above what the national standard is, because 
of their backlog and their particular situation. 

Ms. Berkley. Okay. Mr. Cox, welcome. Are the problems that you 
described in your testimony with training widespread, or more prob- 
lematic to specific regional offices? Is it just across the board? 

Mr. Cox. I think it is across the board. I think the training, that 
there is not consistency with the training. And also, dealing with 
adult learning is a creature unto itself, as I think we are all aware. 

Ms. Berkley. I would hate to have to go to law school again, that 
is for sure. 

Mr. Cox. Yes, I wouldn’t want to go to nursing school again today, 
either. But I think people learn by different modalities. Some do 
well with online-type training. Others need a standard classroom, 
those type things. And so the training that meets the needs of the 
individual in how they learn to perform that job — again, I think one 
very specific thing about these jobs in VBA are you don’t come with 
a ready set of native skills for them. You have to get that training 
there at the job site. I mean, there are people in these VSR positions 
that come with a high school diploma, and some come with graduate 
degrees that do the same work, but they must learn all that work 
there. 

Ms. Berkley. But doesn’t learning have to be somewhat standard- 
ized, the methods — I think in our school systems across the country, 
there are kids that learn different ways, but I don’t think that we 
can provide a specific type of learning path for one student and one 
for another and another. Is there a way that we could standardize 
this nationally, and still be able to provide the training that our VBA 
employees need? 

Mr. Cox. I think that you could get the standardized training. I 
would look at what is done at the VBA Academy in Baltimore. There 
are people — that is their job, to do nothing but training. It is not a 
collateral-type duty or things that they do. They are trainers, and 


20 


they do that work consistently over and over. 

And I do applaud the VBA. I think the training academy in Balti- 
more has done an excellent joh, and will continue to do an excellent 
joh, and that there are good lessons to he learned from the things that 
are housed there. 

Ms. Berkley. Let me ask you a couple of questions — part of your 
testimony concerned me because I am a strong union supporter, and 
a great believer that unions and management working together cer- 
tainly provide a much better product. Can you describe the efforts 
the union has made to engage VBA in collaboration on training and 
performance issues, and has there been a change in behavior? And if 
so, to what do you attribute that change of behavior to? 

Mr. Cox. I would think, you know, also Mr. Walcoff responded 
earlier, and I have always had a great working relationship with Mr. 
Walcoff and his colleagues at VBA, as well as many of us at AFGE. 
However, and I would, you know, say that in his comments, it is “We 
comply with the law, we comply with the contract, we do the things 
that meets our labor-management obligation.” However, many times 
we have to do more. We have a veteran, your constituents, at the end 
of this process, waiting for their claims to be processed, and be pro- 
cessed correctly. So to just comply with the law, with what it says in 
the labor-management relationship, I don’t believe is enough. There 
must be a true partnership, a collaborative relationship of working 
together, and meeting the needs, clearly identifying problems, and 
arriving at a solution, and sometimes sharing information that either 
side kind of holds private. 

Ms. Berkley. Thank you very much. 

Mr. Wise, how does the pass rate of 25 percent and 42 percent for 
the open-book VSR test compare to other similar occupational tests 
with an open-book format? That seems extraordinarily low, if you 
have got the answers at your fingertips. 

Dr. Wise. Well, I would agree. And actually, I am not familiar 
with that many instances of open-book tests, which I think is a fairly 
unique feature of this assessment, in allowing the examinees access 
to the resources, most all the resources they would normally have on 
the job. 

Ms. Berkley. And you know, I found that kind of interesting be- 
cause when I was in college and law school, if I took an open book 
test, I always performed worse, because I think I was expecting that 
I would be able to get the answers really quickly during the test-tak- 
ing time, and it was just a disaster. So I am not sure that is a good 
idea. 

Dr. Wise. Well, it is an important feature of the assessment, that 
in order to ensure its validity we try and replicate the job as closely 
as possible. So for that reason, open book does seem appropriate to 
this exam. 
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Ms. Berkley. Does the low pass-rate suggest that a number of ex- 
perienced employees do not have the requisite knowledge, skills, and 
aptitudes to perform their current johs adequately? And if that is not 
the case, to what do you attribute the low test scores? 

Dr. Wise. The VBA management and, you know, and the experts 
that reviewed it set the pass scores at a level that was adjudged to be 
the point at which people that were just sufficiently qualified would 
need to score, in order to demonstrate the kind of skill that is required 
for this higher-level position. The lower passing rates would indicate 
that many of the candidates who took the exam don’t yet have those 
skills, but they do have the opportunity to go back and study and take 
the exam again. 

Ms. Berkley. Okay, thank you very much. 

Mr. Miller. Thank you very much, Ms. Berkley. 

That is all the questions that I have, too. And everybody’s testi- 
mony has shown that the training claims examiners receive is vitally 
important to achieving VBA’s mission. As I said in my opening state- 
ment earlier, the adjudication of claims is a complex and cumber- 
some process, and I certainly appreciate the importance that Admiral 
Cooper has placed on training by mandating the 80 hours of training 
annually, and hope that the regional offices’ directors are taking heed 
of this mandate. 

I look forward to working with VBA and its stakeholders to en- 
sure that proper training and skills development is at the forefront 
of claims development. I appreciate everybody’s time for coming and 
attending today. Without objection, members will have five legisla- 
tive days to submit materials for the record, as well as post-hearing 
questions to the witnesses. 

With nothing further, this hearing is adjourned. 

[Whereupon, at 3:29 p.m. the hearing was adjourned.] 
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APPENDIX 


Statement of Congresswoman Shelley Berkley 
Subcommittee on Disability Assistance and Memorial Affairs 
Oversight Hearing on the Training Provided to Veterans Benefits 
Administration Claims Adjudicators and the Standards Used to Measure Their 
Proficiency and Performance 
September 12, 2006 

Thank you, Mr. Chairman. VA employees who decide critical 
benefits issues for our Nation’s veterans must have proper training in order 
to fulfill VA’s mission. I am very pleased that we are giving attention to this 
issue. 


I am very concerned that the majority of experienced Veterans 
Service Representatives (VSR) who have taken open book, open resource 
tests were not successful in passing those tests. I am also concerned that 
files reviewed by Committee staff and claims brought to the attention of the 
Committee often contain obvious errors, which VA managers quickly agree to 
correct. 


According to Mr. Walcoff’s testimony, VA provides national 
comprehensive training to regional office staff However, both the employees’ 
union, AFGE, and the American Legion report wide regional office variations 
in training. Having a national plan, does not guarantee that a national plan will 
be effectively implemented. 

The union testimony concerning the VBA’s exclusion of labor 
representatives from meetings about training and performance standards is 
disturbing. Labor and Management need to work together to improve the 
training provided to VA employees and to set realistic performance standards. 

Veterans who have honorably served our nation deserve quality 
decisions in order to receive the benefits and services they deserve. The 
cooperation and collaboration of all partners is essential if this goal is to be 
achieved. 

Thank you, and I look forward to hearing from the witnesses on this 
important subject. 
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U.S, Congresswoman 

Ginny Brown-Waite 

Representing Citrus, Hernando. Lake, Levy, 
Marion, Pasco, Polk, and Sumter Counties 



Statement of Congresswoman Ginny Brown-Waite 
Subcommittee on Disability Assistance and Memorial Affairs 
Opening Statement 

Oversight Hearing on the Training Provided to VBA Claims 
Examiners and the Standards Used to Measure their 
Proficiency and Performance 
9/13/2006 
2:30 PM 

Thank you Mr. Chairman, 

I would also like to thank all of the witnesses for appearing 
before the committee today. 

The Veterans Benefits Administration’s claims process is 
arguably the backbone of the Department of Veterans 
Affairs (VA). To provide our nation’s veterans with 
benefits, the VA requires qualified claims examiners. With 
the number of annual claims filed by veterans continuing to 
rise, examiners have an extraordinary responsibility before 
them. Long wait times or mistakes are unacceptable, 
especially for those veterans who might not have much 
time left. 
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I applaud the VA’s recent efforts to improve the education 
of its claims examiners. Job-related education will go a 
long way toward ensuring that our nation’s veterans receive 
their benefits in a timely fashion. 

As Members of Congress, we have an extraordinary 
responsibility to veterans. These individuals deserve the 
timely and efficient adjudication of their claims. I look 
forward to working with my colleagues on the Committee 
to see that this occurs. 

Thank you. 
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Mr. Chairman and members of the Subcommittee: Thank you for the 
opportunity to appear before you today to address training for Veterans Benefits 
Administration (VBA) claims adjudicators; the standards we use to measure their 
proficiency and performance; and how we communicate changes in laws, 
regulations, policy and procedures to field station staff. I am accompanied by 
Mr. James Whitson, Director of VBA’s Eastern Area Office; Ms. Janice Jacobs, 
Deputy Director of VBA’s Compensation and Pension Service; and Ms. Dorothy 
Mackay, Director of VBA’s Office of Employee Development and Training. 

Training 

Training is central to every quality organization. VBA is committed to 
ensuring all employees have the opportunity to learn and develop the knowledge, 
skills, and abilities required to be successful in their current roles and to prepare 
them for positions of increasing responsibility and leadership. We therefore 
recognize training as a core element of VBA’s infrastructure and key to our 
succession planning efforts. Ensuring an effective training program is especially 
critical as we hire new employees to replace many of our more experienced 
employees who are expected to retire over the next few years. 

VBA has deployed new training tools and programs to provide a more 
consistent approach to training. Newly hired veterans service representatives 
(VSRs) receive a comprehensive and consistent foundation in claims processing 
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principles through a national centralized training program called “Challenge.” 
After this initial centralized training, employees follow a national standardized 
training curriculum at their home regional offices. This 23-week curriculum 
includes full lesson plans, handouts, student guides, instructor guides, and slides 
for classroom instruction. A consistent, nationally developed training program 
also is provided to newly appointed Rating VSRs (RVSRs). 

Additionally, standardized computer-based tools have been developed for 
training decision-makers. The Training and Performance Support System or 
TPSS provides cooperative-learning modules centered on the skill sets required 
to become a fully functioning VSR or RVSR. These TPSS modules are also 
used as refresher training for experienced decision-makers. Currently, there are 
27 TPSS modules available covering the claims and appeals processing 
functions of VSRs and RVSRs, with additional modules being developed. These 
modules are regularly updated and supplemented with on-the-job training at the 
regional office level, as needed. 

The Electronic Performance Support Systems or EPSS provides VSRs and 
RVSRs with electronic job aides for immediate reference as they work cases. By 
responding to a series of questions based on the issue presented, EPSS guides 
claims adjudicators in the decision process. EPSS also offers tools that explain 
medical terminology and describe medical conditions to help decision-makers 
understand the severity and impact of claimed disabilities. 

VBA also provides training to claims adjudicators through distance-learning 
technology. Satellite broadcasts are produced throughout the year on topics 
covering the full spectrum of issues including legislative and policy changes, the 
impact of court cases, and rating considerations for particular disabilities. 

To ensure the trainers have the skills to make their training classes effective 
and engaging, VBA provides a week-long Instructor Development Course. All 
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trainers for national level training programs, such as “Challenge” training, must 
graduate from the Instructor Development Course. 

An extensive training website provides another ready-reference tool to assist 
regional office employees. The website offers lesson plans, training guides, job 
aids, and other training materials to assist newly hired or journey-level VSRs and 
RVSRs. The website also contains links that cross-reference documents and 
web sites to reduce VSR or RVSR research time. 

This fiscal year, the Under Secretary for Benefits established a policy 
requiring that regional offices provide all claims adjudicators with a mandatory 70 
hours of job-specific training. Most other employees are required to be provided 
with at least 40 hours of training. In FY 2007, the mandatory training for claims 
adjudicators will Increase to 80 hours. 

VBA has established core requirements for this technical training, with 
Headquarters’ business lines and staff offices required to identify curricula for 
each major position in VBA. To ensure training is properly planned and 
executed, each regional office is now required to develop an annual training plan 
and report twice yearly on its training accomplishments. Regional office training 
plans and reports are reviewed in Headquarters to identify trends, training gaps 
(where more centrally provided content is needed), and local best practices that 
could be adapted for national use. Every facility receives written feedback on its 
training plan and progress. This process has resulted in enhanced consistency 
in training and has produced data on training trends, strengths, and areas for 
improvement. This process provides regular snapshots of the state of training in 
VBA and has increased accountability for ensuring employees receive quality 
training. 
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Dissemination of New Information and Changes to Field Employees 

VBA ensures that regional offices are provided timely, accurate, and complete 
information on regulatory, statutory, and administrative changes, as well as any 
legal or judicial decisions that affect claims processing for our nation’s veterans. 
This includes changes in policy and procedures that affect the way VBA does 
business, such as what to include in a letter informing a veteran on what is 
needed to substantiate his/her claim. 

Field station employees receive this information through various delivery 
methods. The Compensation and Pension (C&P) Service issues guidance 
letters (called “Fast Letters’’) to advise field employees on policy and procedural 
changes and legal revisions. In 2005, the C&P Service issued 23 Fast Letters; 

16 have been issued to date in 2006. 

Decision Assessment Documents (DADs) are prepared following a precedent 
decision of the Court of Appeals for Veterans Claims (CAVC) or decision of the 
U.S. Court of Appeals for the Federal Circuit (Federal Circuit). DADs are also 
issued on precedent opinions of the Office of General Counsel. DADs explain 
the judicial holding, summarize the facts and reasoning of the particular case, 
and inform field employees of any impact on VBA. A DAD is typically prepared in 
the days following a court decision and then sent to field employees via 
electronic mail and posted on the C&P Service’s website. 

Additiorially, the C&P Service maintains a separate page on its website 
specifically designed to encourage two-way communication between claims 
adjudicators and the C&P Service staff. There is a “Frequently Asked Questions’’ 
page that provides answers to questions posed by field station employees that 
were unable to be resolved at the local level or that have national policy 
implications. Questions are answered by subject matter experts who serve on a 
“Q&A" committee. Answers are categorized and indexed according to subject 
matter and date posted. 
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The C&P Procedures Manual is continually updated to incorporate guidance 
and changes communicated through Fast Letters, DADs, satellite broadcasts, 
etc. Notification of manual changes are sent to field employees by electronic 
mail and also posted on the C&P Service’s website. 

The C&P Service also communicates extensively with the field through 
Veterans Benefits Network (VBN) broadcasts. These broadcasts are designed to 
provide in-depth analysis and discussion on VBA procedures and policies by 
subject matter experts. They are generally taped, but frequently also transmitted 
live so field employees can call in questions or request on-the-spot clarification. 

In recent months, VBN has aired broadcasts on the Manual Rewrite Project and 
on the development of claims for post-traumatic stress disorder, which included 
practical case scenarios. 

The C&P Service hosts monthly conference calls with veterans service center 
managers to discuss current “hot” topics. These calls provide an opportunity to 
clarify new and existing policies and procedures and address recent court cases. 
These live conference calls give field stations an opportunity to pose questions 
and concerns directly to the C&P Service for immediate feedback. 

National Performance Standards for VSRs and RVSRs 

Another VBA organizational cornerstone to improve the delivery of benefits 
and enhance accountability is our system of individual performance assessment. 
All VSRs and RVSRs are subject to national performance standards that are 
reviewed periodically and amended as necessary in response to changes in 
workload and claims processing procedures. Managers use an automated tool 
called ASPEN to track work items completed and measure VSR and RVSR 
performance. Local accuracy reviews are conducted for all decision makers 
using the national qualify criteria (Systematic Technical Accuracy Review or 
STAR). 
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The genesis of the current performance standards for VBA’s claims 
adjudicators was the review by former Secretary Principi's Claims Processing 
Task Force, which was chaired by Admiral Daniel L. Cooper prior to his 
appointment as Under Secretary for Benefits. The Task Force provided many 
recommendations to improve the efficiency and effectiveness of VBA claims 
processing, including reengineered organizational structures and business 
processes for adjudicating veterans' compensation and pension claims. This 
new claims processing structure, called the Claims Processing Improvement 
(CPI) Model, placed greater emphasis on consistency of process and 
accountability for results, and led to development of new national performance 
standards. 

Teams were chartered and charged with creating national VSR and RVSR 
standards. The standards focused on the key elements of quality, productivity, 
customer service, and timeliness. The performance standards were tested at 
several pilot sites and revised based on feedback from those tests. Agreements 
were reached with the American Federation of Government Employees (AFGE) 
prior to implementation. The current VSR performance standards have been in 
effect since October 2005. A team of regional office directors, service center 
managers, rating coaches, and Headquarters staff is currently considering 
revising the RVSR standard based on recent business process changes, such as 
the consolidation of the pension maintenance and Benefits Delivery at Discharge 
workloads. 

VBA’s four Area Offices and the Office of Field Operations monitor 
performance at the regional office, area, and national levels for accountability 
purposes, as well as for performance trends. 
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Skills Certification Testing 

In conjunction with the national performance standards, VBA has developed a 
certification process to assess job proficiency. By successfully demonstrating job 
proficiency through the certification process, an employee is promoted to the 
journey level, thereby linking job proficiency to pay grade. Through the national 
certification program, VBA is raising the skill levels of our core decision-makers 
and producing greater consistency in claims decisions. 

The concept of skills certification originated in the late 1990s as part of VBA’s 
Business Process Reengineering (BPR) effort. However, the initiative did not 
begin in earnest until January 2000 when an MOD was signed with AFGE 
announcing the adoption of a certification program for the VSR, RVSR, and 
Decision Review Officer (DRO) positions. The VSR position was selected for the 
pilot program. 

In December 2000, VBA contracted with the Human Resources Research 
Organization (HumRRO), a nationally known research group that develops 
products and provides services to improve organizational performance, to assist 
in the development of a certification instrument. Since that time VBA subject 
matter experts have been working with HumRRO to design, develop, and 
validate an effective certification process that assesses the knowledge and 
readiness of VSR GS-996-10 incumbents for promotion to the GS-11 level. A 
mid-course review was conducted following the implementation of the CPI Model 
to ensure the validity of the certification instrument with respect to the new 
organizational structure and processes. 

Prior to VSR Skills Certification, the career ladder for the VSR position ended 
at the GS-10 level. The higher level award review and authorization functions 
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were performed by GS-996-1 1 Veterans Claims Examiners (VCE). Employees 
were selected for these VCE positions through merit promotion. 

Beginning in late 2002, VSR job announcements began identifying the full 
performance level for VSRs at the GS-1 1 level, contingent upon successful 
completion of certification testing. Through successfully passing the certification 
test VSRs demonstrate that they have the skills necessary to perform the full 
range of VSR duties, including independently working the most complex cases 
and reviewing and authorizing the work of others. 

The certification process consists of a 100-question multiple-choice, open- 
book test given over the course of two sessions, one in the morning and one in 
the afternoon. VSRs have 2.75 hours to complete each session with an hour 
break between sessions. The test design is based on a blue print of knowledge 
areas developed by subject matter experts. VSRs are provided access to on-line 
references and invited to bring any other reference material to the test site, 
including individually prepared notes. To pass the test, VSRs must achieve a 
minimum overall score and a minimum score on the compensation portion of the 
test. 

The first validation test was performed in August 2003. There were 298 
participants in the first test. Of these, 75 passed for a pass rate of 25 percent. 
While this was a lower national pass rate than anticipated, we were confident that 
the instrument would advance only those with the skills and abilities to perform at 
the GS-1 1 level. In many cases, the test results were consistent with the 
predictions of local managers as to who would pass and who would not. 
Modifications were made to the instrument and test administration process to 
address some of the validation test findings. As an example, the time allotted for 
taking the test was increased, as was the break period between the morning and 
afternoon session. 


8 


33 


VBA conducted a second validity test in April 2004. There were 650 
participants in the second test. Of these, 188 passed for a pass rate of 29 
percent. Despite the continued low pass rate, HumRRO found a significant 
correlation between performance on the test and an assessment of on-the-job 
performance, further confirming the validity of the test instrument. Following 
each administration of the test, AFGE was provided with a copy of the reported 
findings and analysis produced by HumRRO, along with informational briefings. 

With low pass rates on the first two validation tests, a 20-hour VSR 
“readiness” training curriculum was developed to prepare VSRs for certification 
testing. The training is mandated for all VSRs who volunteer to sit for the test. 
We believe the combination of local and centralized training programs, along with 
certification “readiness” training, will not only improve pass rates, but will result in 
a work force better prepared to provide quality service to veterans. 

A third test was administered on May 3, 2006, to 934 VSRs nationwide. 
During administration of the afternoon portion of the test, a problem was 
discovered. Multiple test items from the morning portion of the test were 
duplicated on the afternoon version. After consultation with HumRRO, it was 
decided that the best way to ensure a fair assessment of the candidates’ ability to 
perform at the GS-1 1 level was to have the candidates complete the test items 
that should have been delivered during the afternoon session. On June 7, the 
correct afternoon version of the test was administered. The May/June 
administration of the test resulted in an improved pass rate of 42 percent. 

VBA has to date promoted 633 VSRs to the full-performance GS-1 1 level 
through the certification testing process. We are confident that we have 
developed a valid assessment tool and certification process to raise the skill level 
of our VSRs. Both the instrument and process are constantly reviewed, updated, 
and adjusted to ensure the right skills are being tested and that employees who 
are successful on the test are successful on the job. We are in the process of 
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expanding skills certification testing to the RVSR position for those seeking to 
progress from the GS-1 1 level to the GS-12 journey level. 

Mr. Chairman, this concludes my testimony. I greatly appreciate being here 
today and look forward to answering your questions. 
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Good afternoon. I am Lauress L. Wise, the president and CEO of the Human Resources 
Research Organization, known less formally as HumRRO. HumRRO is a non-profit, 501 (c)3 
research and development organization, established in 1951, that works with government 
agencies and other organizations to improve their effectiveness through improved human capital 
development and management. 

I have been asked to testify today about work that HumRRO has done for the Veterans 
Benefit Administration (VBA) on their program for certifying essential skills for Veterans 
Service Representatives. These service representatives play a key role in seeing that our veterans 
receive the fall array of benefits to which they are entitled. VSR performance at the highest level 
of the position requires a thorough understanding of an extensive set of policies and procedures 
concerning veterans’ benefits and skill in identifying appropriate applications of these 
procedures to individual circumstances. The skills certification program embarked on by die 
VBA is critical to ensuring that service representatives have the knowledge and skills needed to 
perform their jobs effectively. 

Development of the VSR Skills Certification Test 

In January 2001, the VBA contracted with HumRRO to assist in the design, development, 
and validation of an effective and defensible certification process for the VSR position. 

HumRRO has worked with VBA to develop a certification program that assesses the knowledge 
of GS-996-10 incumbents to judge their readiness for promotion to the GS-1 1 position. GS-10 
VSRs who pass the certification test are promoted to the GS-1 1 position; GS-1 Is who pass the 
test receive a bonus. 
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Jo^ Analysis 

During 2001, HumRRO conducted an extensive analysis of the VSR job. We worked 
with senior incumbents to identify critical VSR tasks, rate their importance, and identify the 
knowledge and skills needed to perform these tasks effectively. The critical tasks were organized 
into functional areas identified as important by the VBA Design Team'. These areas included: 

(a) Compensation, (b) Pension, (c) Public Contact, (d) Administrative Decisions, and (5)Appeals. 

Development of Test Questions 

The Design Team used the results of the survey to develop a test blueprint, which 
specified the number of test questions needed to cover each of the functional areas. We then 
trained the Design Team to write high quality test questions (items) and conducted several item 
development workshops to review and revise these questions. HumRRO worked with the VBA 
to conduct a pilot test of the test questions and test administration procedures. Many questions 
were dropped after the pilot test either because the item statistics were less than optimal or 
because pilot test participants indicated problems with a question. This is the norm; we typically 
develop about three times the number of items we need for administration, knowing from 
experience that we will lose over half in revision or piloting. 

Changes to the VSjRJoh 

When the Claims Processing Task Force Report was published in the Fall of 2001, the 
certification program was put on hold while recommendations from the report were put into 
place. The Claims Process Improvement (CPI) initiative that followed included some significant 
changes to the VSR position. In April 2002, VBA contracted with HumRRO to conduct several 
site visits to determine whether the test items, which had been written at a time when the VSR 
job was a generalist position, were still appropriate for VSRs who were now working on 
specialized teams. 

Following the site visits, HumRRO met with representatives of VBA, Compensation and 
Pension (C&P) training, and the VBA Central Office to discuss the impact of CPI on training 
and skills certification. The decision was made to proceed with the generalist test because the 


' The VBA Design Team represented all major stakeholders in the claims processing field (i.e., VBA management, 
AFGE, the Compensation and Pension line of business, incumbents, and veterans service organizations) 
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policy was that VSRs would be rotated across teams to maintain the skills acquired in training. A 
GS-1 1 in this position can be assigned to any team based on the needs of the station and small 
stations may only have one or two GS-1 Is. These GS-1 Is must be capable of reviewing and 
authorizing all of the work performed at the station regardless of the team from which it 
originated. Specialized tests reflecting specific team assignments would not tap skills that would 
be needed for future assignments, so HumRRO recommended that work continue using the 
general test blueprints previously established. 

Sestarting the ft-ogram 

In the Fall of 2002, VBA put together a new Design Team whose task it was to get the 
certification process moving again. The Design Team reviewed the test blueprint, the Candidate 
Guide, Test Administrator Manual, and other test support documents (e.g., background 
information forms, confidentiality agreements). These support documents were updated to 
reflect changes in the program in the intervening years. The Design Team also reviewed the test 
questions and dropped some due to changes in the VSR job. They also wrote new questions to 
take the place of those that were dropped. These new items were pilot tested In February, 2003 
in preparation for a spring test. This pilot test used the updated support documents, which would 
also be used in that test. 

Operational Fiehl Test 

An operational field test was conducted in August 2003 that involved administering an 
over-length version of the skills certification test to 298 eligible GS-10 and GS-1 1 VSRs. The 
operational exam is designed to include 100 operational questions; we administered two over- 
length exams (about 120 items each) to allow us to collect data on all the items in the item bank 
so they would be ready for use in future administrations. HumRRO staff identified a set of 100 
questions for each of the two forms that met the test specifications and demonstrated solid 
statistical properties, and computed overall scores based on the selected items. 

Passing Score 

After the field test was completed, subject matter experts (senior VBA employees who 
had been promoted from the GS-1 1 position and were “Super Senior” VSRs or Ratings VSRs) 


Lauress L Wise, HumRRO 


Page 3 


39 


HumRRO Testimony 

participated in a workshop to establish a minimum passing score for the test. HumRRO used an 
established standard setting procedure that required experts to estimate, for each question, the 
percent of minimally qualified examinees who would answer the question correctly. 

To pass the certification, candidates had to pass the whole test, as well as meet minimum 
score requirements on the compensation and pension subtests. Based on the standard setting 
results, candidates were required to correctly answer about three-quarters of all of the questions, 
three-quarters of the compensation questions, and just over half of the pension questions to pass 
the exam. Seventy-five candidates (25%) passed all three hurdles. While VBA had hoped for a 
higher pass rate, they verified with management at several Regional Offices that candidates who 
passed were those who were expected to do so, and those who failed were expected to have 
difficulty meeting the certification requirements. 

Subsequent to the field test, test blueprints were revised giving more emphasis to 
compensation and less to pensions. Another standard setting workshop was held to establish 
minimum passing scores for the first operational administration in May 2006. Based on results 
from this workshop, candidates were required to correctly answer two-thirds of all questions and 
also two-thirds of the compensation questions to pass the test. The separate requirement based on 
pension questions by themselves was dropped. 

Criterion-Xelated VaUdation Study 

In 2004, the Office of Personnel Management reviewed the VSR Skills Certification 
Program to determine whether there were potential problems with using it as part of the 
promotion process. The overall passing rate in the field test was generally low, about 25%. A 
particular concern was that the passing rate for African Americans was significantly lower than 
for other incumbents. When a test results in this type of adverse impact for a particular group, 
legal guidelines require employers to demonstrate that test scores are a valid reflection of the 
skills needed to perform the job. While HumRRO had previously collected content validity data 
showing the relevance of each of the test questions, the VBA decided to further strengthen the 
validity claims for the test and asked HumRRO to conduct a criterion-related validation of the 
test. 

The field test relied on content validity as the basis for establishing a relationship to the 
VSR position. Content validity asks the question; How well does the assessment sample the 
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range of important tasks, behaviors, or knowledge associated with effective job performance? 
Legal and professional authorities (Uniform Guidelines on Employee Selection Procedures; 
Equal Employment Opportunity Commission, 1978 and the Principles for the Validation and 
Use of Personnel Selection Procedures; 4th ed.. Society for Industrial and Organizational 
Psychology - SIOP, 1999) have converged on several basic principles for content validation 
studies. Evidence for content validity comes from following well-established and accepted job 
analysis and test development steps and from data that demonstrate a direct link between the 
selection procedures and job requirements. This is accomplished by: (a) detailing job tasks and 
the knowledges, skills, and abilities (KSAs) required to perform those tasks; (b) establishing 
linkages between the job tasks and KSAs and, (c) demonstrating linkages between KSAs and test 
content areas. In developing assessment instruments, including certification tests, it is 
HumRRO’s practice to follow the guidelines for establishing the content validity of a test even if 
we plan to use a criterion-related validation strategy, so we had already done the work to 
establish content validity. 

While content-related validation is established through expert judgments, evidence for 
criterion-related validity consists of demonstrating a useful relationship between a selection 
procedure (predictor) and one or more measures of job performance (criteria). This is 
accomplished by administering the predictor tests (i.e., the certification test) to candidates and 
gathering information on how these individuals perform on the job. Ideally, we would find that 
individuals who score higher on the tests are those persons who perform more effectively on the 
job, while individuals who score lower on the tests perform less effectively on the job. The 
Principles for the Validation and Use of Personnel Selection Procedures (Society for Industrial 
and Organizational Psychology, 1999) outlines several conditions that should be met before 
proceeding to conduct a criterion related validity study. They are as follows: 

1. Criterion related validity studies should be conducted for jobs that are rea.sonably 
stable and are not in a period of rapid evolution. 

2. Relevant, reliable, and uncontaminated criterion measures against which to 
validate the predictor tests are essential for successful criterion related validation 
studies, 

3. The sample on which data are collected should be reasonably representative of the 
population to which the results are to be generalized. 
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4. A criterion related validity study should have adequate statistical power to yield a 
significant predictor-criterion relationship, if one exists. Factors affecting 
statistical power include sample size, degree of variability in the predictor (i.e., 
certification test score), reliability of the criterion, etc. 

At the time of the validation study, revisions to the VSR position under the CPI model 
had been in effect at VBA for over a year, and all candidates for certification had been on the job 
for at least one year. Incumbents had sufficient time to acclimate to the job redesign, so the job 
was considered stable. We developed a performance measure that combined existing data on 
productivity and quality with supervisor ratings of performance. This measure met the criterion 
for relevance described in point 2 and demonstrated sufficient reliability. The sample on which 
the data were collected included almost 700 candidates, so the sample size was adequate to 
generalize to the general population of GS-10 VSRs. These factors made criterion-related 
validity an appropriate strategy for the VSR Certification Test. Results of the criterion-related 
validity study indicated a strong statistical relationship between scores on the certification test 
and the measures of job performance. 

The May 3, 2006 Test Administration 

The first regularly administered test for the Veterans Service Representative (VSR) 
Certification Program was conducted May 3, 2006. Stations that could not accommodate all 
candidates in one day also tested on the following two days, as necessary. The test was 
administered to 934 candidates. Two forms of the test were administered so that different 
examinees did not necessarily get the same questions in the same order. Each test form included 
100 scored questions and 20 additional questions being pilot tested for future use. In the May 
2006 test, the two test forms had 67 operational items in common, albeit in different locations 
within the test. Because of the length of the test, the test is split across two sessions— morning 
and afternoon. Candidates received a separate booklet for each session. 

Due to a processing error at HumRRO, some of the questions in the afternoon booklets 
were inserted into the incorrect forms. This error resulted in duplicating some questions from the 
morning session in the afternoon session booklets for the corresponding test form. Quality 
control procedures in effect at the time included a review of each test booklet, but did not include 
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a comparison of the morning and afternoon booklets for each test form. Consequently, the 
processing error was not caught prior to the test administration. 

Calls from the field alerted VBA and HumRRO to a potential problem. HumRRO staff 
investigated to find out how widespread the problem was and alerted VBA to the extent of the 
problem. Thirty-three items had been duplicated on one form and 34 on the other. The VBA 
Eastern Area Director, Jim Whitson, set up a teleconference with the HumRRO Project Director 
and the management members of the VSR Design Team. Subsequently, he sent an 
announcement to all stations advising the candidates to continue taking the test with the duplicate 
items assuming that all items would be scored, and that an equitable solution to the problem 
would be identified as quickly as possible. VBA also made the decision to continue the test as 
scheduled on the following days, instructing candidates to answer the duplicate items as carefully 
as though they would be scored. While we had not determined a plan of action, it was possible 
that we would decide to score the duplicate items, so it was important that candidates answer 
those items to the best of their ability. 

Ifow The I^od/em Was ffand/ed 

On May 1 1, 2006, HumRRO Vice President Beverly Dugan, VSR Certification Project 
Director Patricia Keenan, and I met with VBA leadership to discuss the problem and identify 
possible methods of providing valid scores to participants. Our discussion identified several 
possibilities, including using some of the pilot items to construct an 80-item test, ignoring the 
redundancy and scoring each of the duplicate items to provide a 100-item test, and conducting a 
supplemental administration using the items that were originally intended to be included in each 
of the afternoon test booklets. 

The solution agreed upon was to conduct a supplemental test and administer the items 
that were originally intended to be presented in each of the afternoon tests. This allowed 
everyone to be scored on 100 separate items, kept the test mapped to the blueprint exactly as 
designed, and made the May 2006 administration much more equivalent to the operational field 
test and the validity test, and to those planned for the future. 

The supplemental test was held on June 7, 2006. A total of 46 people who took the May 
test chose not to sit for certification in the supplemental test; all individuals who chose not to 
take the supplemental test had failing scores based on the items they did take. The original and 
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HumRRO Testimony 

supplemental test questions were scored as intended and 370 (42%) of those who took the entire 
test passed. The supplemental testing created some inconvenience to the examinees and 
additional burden to those who administered the tests, but the end result was an assessment that 
covered the content framework as intended with questions and scores that were psychometrically 
sound. 

Contributing Factors 

Several factors contributed to the error in assembling the May 2006 test booklets. One 
such factor was the limited time available for assembUng and checking the test booklets. The 
VSR job continues to evolve. For example, new types of cases are often added to the caseload, 
new electronic tools and databases are developed, and more pension cases are being moved to 
Pension Maintenance Centers. In addition, one of the prime references, M21-1, is undergoing a 
major revision, HumRRO must rely on expertise of VBA staff members to consider how each 
new change might affect the validity of the test questions in the VSR certification item bank. A 
workshop to review test questions was held in April. The item writers reviewed all of the items, 
revised many of them, and updated the references. Following the workshop, HumRRO staff 
implemented the edits to the item bank. The revisions were more extensive than anticipated and 
the work was completed late in the week prior to the scheduled packing date. We had only two 
days to select the items and put together the four test booklets. The item selection was made 
more difficult by the fact that, in the two years since the previous administration, many items had 
become outdated, requiring revision and a new field test, so there were a limited number of 
remaining items to choose from in some areas of the blueprint. In retrospect, it was clear that 
more time was needed for assembling and checking the test forms. 

HumRRO staff members routinely check test booklets for potential problems (e.g., stray 
marks from the printing process, items split across pages). We did not explicitly compare 
morning and afternoon versions of the test, which was the only way to have identified the 
problem. Additional review by VBA experts would be required to provide one additional check 
of the technical accuracy of each question and the correctness of the scoring key. While scoring 
was not an issue in the May administration, it is clear that a more definitive process for final 
technical review of each test form is needed. 
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Preventing the I^oblem in the Future 

First, we have expanded final test form quality control procedures to review the morning 
and afternoon booklets for each form together. In addition, to relieve the time problems 
experienced in May 2006, we have changed the timing of the item writing workshops to provide 
more time after the workshops for assembling and checking operational test forms. 

The second problem, the need for a definitive review by VBA experts, will be solved by 
including reviews of the test items and booklets by Compensation and Pension (C&P) Services 
staff at VBA. HumRRO will identify the test items to be included in the test and send them to 
C&P to review the items, keyed responses, and references. After that review, HumRRO will 
make any needed edits, put together the actual test booklets and send them to C&P for a final 
review. We implemented this procedure for the August 9, 2006 test and there were no problems 
with the test. 

Summary and Conclusions 

The VSR certification test is an important tool for improving the effectiveness of the 
VSR workforce in serving the benefit needs of our veterans. The testing process is based on a 
solid analysis of the VSR position and questions were developed and mapped to an established 
blueprint derived from that analysis. The validity of the test scores for making promotion 
decisions is supported by both content-related and criterion-related validity evidence. 

A number of factors contributed to an error in assembling test booklets for the May 2006 
administration of the VSR certification test. Once discovered, corrective action was taken that 
led to appropriate scores computed from test questions matching the design blueprint exactly. 

We have no reason to question the validity of these scores. Test assembly and review procedures 
have been expanded to preclude similar errors with future test forms. 
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Dear Chairman and Members of the Subcommittee: 

The American Federation of Government Employees, AFL-CIO, which 
represents more than 600,000 federal employees who serve the American 
people across the nation and around the world, including roughly 150,000 
employees in the Department of Veterans Affairs (VA), is honored to testify today 
regarding the training provided to the Veterans Service Representatives (VSRs) 
and Rating Specialists who adjudicate claims for the Veterans Benefits 
Administration (VBA), and the standards used to measure their proficiency and 
performance. 

In my capacity as 1®' Vice President of AFGE’s National Council and a 
local union president, I have visited the vast majority of VBA Regional Offices 
(ROs) around the country. VSRs and Rating Specialists everywhere are 
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concerned. They share the Subcommittee’s concerns about inconsistencies in 
decisionmaking. They are equally concerned about the training deficiencies and 
unrealistic performance standards that contribute to these inconsistencies. The 
Government Accountability Office (GAO) and the VA Office of the Inspector 
General (IG) have both linked training deficiencies to backlogs and decision 
inconsistencies. The IG even found a direct correlation between the frequency of 
classroom instruction and payment levels. If training were standardized and more 
uniform, VBA would be better able to do a root cause analysis of geographic 
variations in disability awards. 

Training is an investment worth making, and we greatly appreciate this 
Subcommittee focusing on it at today’s hearing, as well as the Committee’s 
Fiscal Year 2007 budget recommendations for more VBA adjudication staff and 
training resources. 

When I go out in the field, I see employees working evenings and 
weekends without compensation in order to try to keep up with their cases. I see 
employees meeting on their lunch hours to familiarize themselves with new laws 
and regulations in the absence of more formal sessions set up by management. 

1 see VSRs^ feeling anxious about passing the skills certification test 
because they were not adequately informed about what to study in order to 
prepare for the test, or what proficiency level is needed to pass. VSRs and 
Rating Specialists are very dedicated to getting every veteran an accurate, timely 
claims decision. In some offices, half the employees are veterans and many are 
themselves service-connected. 

‘ Only VSR GS-lOs are eligible to take the skills certification test at this time. 
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In my testimony today, I hope to give you helpful examples of problems 
occurring out in the field as \well as suggestions for improvement. I also want to 
point out what is working well. However, my ability to report from the field or 
comment on management action is severely limited by VBA's refusal to share 
information with us. We hear many troubling reports but we cannot measure the 
full extent of the problems with training and performance standards. 

More generally, we are concerned about management’s increasing refusal 
to collaborate with us about training and performance standards. For example, 
AFGE initially worked with VBA on the development of the Claims Process 
Improvement (CPI) Model, and reached an agreement with them regarding the 
recommendations of the CPI Task Force. Even though VBA has since made a 
number of revisions to the CPI model, such as removing Rating Specialists from 
Pre-Determination Teams and centralizing the Public Contact Unit, employee 
representatives were excluded from recent Task Force meetings. 

We believe that all stakeholders - including employee representatives and 
veterans’ groups - should have input into the design of training programs, and 
information about how well these programs are working in different ROs around 
the country. Similarly, stakeholders need a voice in developing and monitoring 
the certification testing process and production quotas. Flawed training and 
performance standards lead to processing delays and more appeals, and most 
important, impact every veteran filing a claim with VBA. 

As a registered nurse, I can say firsthand that there is a vast difference 
between the training needs of a VHA medical professional and a VSR at VBA. 


{0022353i.DOC) 


3 


48 

While a nurse or doctor can perform effectively at a VA hospital after orientation, 
virtually all the skills of a claims adjudicator must be acquired on-the-job. By 
VBA’s own estimates, it takes at least two to three years for a new VSR to 
operate at a fully productive, independent level - whether he or she comes to 
VBA right out of high school or after graduate school. VSRs with fewer than five 
years of experience need ongoing training and intense supervision to become 
facile enough to achieve high levels of accuracy and customer service. That is 
why VBA called on its most experienced rating specialists when it formed the 
Tiger Team in 2001 to reduce backlogs of older veterans’ cases. A shortage of 
experienced employees will only become more pronounced as senior VSRs and 
Rating Specialists retire and current policies cause frustrated, newer employees 
to leave VBA within a few years of arrival. 

RECOMMENDATION: A joint VBA-stakeholder team should develop a 
national training plan. 

As VBA testified before this Subcommittee last fall, training is central to 
every quality organization. We are pleased with VBA's efforts to develop new 
training tools and centralized training programs, but much more needs to be 
done to ensure that quality and consistent training is provided to every VSR and 
Rating Specialist. VBA training operates much more like national guidance than 
a national training plan, resulting in tremendous variations in quality between 
ROs. 

A joint labor/management team should be established to develop a formal 
national training plan with clearly defined curriculum. In his response to the 
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Inspector General's report on state variations in awards, the Under Secretary for 
Benefits concurred with the need for centrally developed and directed training. 
The current training directives are too vague. For example, to provide VSRs with 
refresher training on “Duty to Assist", a critical issue that was the subject of 
litigation, the only guidance given to the trainer is a set of four topic bullets and 
the requirement that it last’ one hour. 

A Joint labor/management team would also be able to identify best 
practices from local facilities, and regularly assess the quality and thoroughness 
of local training programs. 

This plan should mandate that every employee receive roughly the same 
amount of training under defined timeframes. Currently, VBA requires that VSRs 
receive a set number of hours of training each year. However, mandatory training 
in areas other than benefits, such as ethics, privacy and sexual harassment also 
count toward that hours requirement. Thus, when it comes to finding time for 
training on benefit programs, supervisors facing tremendous pressure to clear 
backlogs are likely to sacrifice training for production numbers. Training on 
computer help aids is often cut short also. 

Another common inconsistency is that older employees are much less 
likely than newer employees to receive instruction under the recently Training 
Performance and Support System (TPSS) tool. 
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RECOMMENDATION: Rotation of VSRs through all claims processing 
teams should be mandatory and monitored. 

More and more employees are losing training opportunities because their 
supervisors are not rotating them through all the relevant teams. (VSRs work in 
four teams: Pre-Determination, Post-Determination, Triage and Appeals.) This 
impedes their ability to handle a full array of claims effectively and lessens their 
chances of passing the skills certification test. Some VSRs have worked on only 
one team during their entire tenure at VBA. 

RECOMMENDATION: VBA should develop a cadre of effective, competent 
trainers with formalized training skills and adequate subject matter 
expertise. 

Trainers are currently selected without well-defined criteria, resulting in 
great variations in the quality and thoroughness of the training. The typical trainer 
is a mid-level or senior VSR who has not had formalized instruction on training. 
Some employees go to the VBA Academy where they are more likely to learn 
from experienced trainers; others never go to the Academy. For skills 
certification training, some VSRs were taught by the same person who designed 
the test (presumably well-versed in what to study), while others did not have this 
advantage. We received reports that some trainers did not even know what the 
test was going to cover! 
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RECOMMENDATION: Continuing education should be mandatory and the 
curriculum and schedule should be set nationally. 

VSRs and Rating Specialists must stay updated on a steady stream of 
new laws, regulations and court decisions. They need timely, well-developed, 
refresher courses to ensure that they understand the impact of these changes 
and implement them correctly and uniformly across offices. Managers should set 
aside specific, regular times for current employee training. Employees tell us that 
in the past, it was standard practice to have regular end-of-the-week meetings to 
learn about new development and go over significant cases. Where this practice 
is still in place, such as the Public Contact Unit in Roanoke, Virginia, the 
employees find the meetings to be a valuable learning tool. Clearly, elimination of 
this practice is another byproduct of productivity pressures on the workforce. 
RECOMMENDATION: Oversight of VBA’s training program should be 
formalized, ongoing and transparent 

As already noted, all stakeholders should be able to monitor the 
effectiveness of VSR and Rating Specialist training. Veterans’ groups and 
employee representatives are on the front lines, and therefore, are in an 
excellent position to identify best practices and areas of weakness. Given the 
vast subject matter that needs to be learned, and the number of offices involved, 
a formal quality assurance program for VBA training is especially important. The 
oversight process should also allow require regular reports to Congress. 
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RECOMMENDATION: Current performance standards should be 
revised to enable VSRs to adequately develop claims and receive needed 
training. 

Pursuant to an agreement between VBA and AFGE, national performance 
standards to boost VSR productivity were put in place in 1997. They were 
revised by agreement in 2005. These standards set a national floor and each 
RO is free to set them higher. Prior to the creation of national standards, each 
station set its own production quotas. 

The national standards continue to need revision. The share of employees 
who are meeting the standards is inexplicably low, calling into question the 
validity of these measures. Approximately a third of those evaluated around the 
country are failing to satisfy the standards, on the basis that they are not meeting 
production quotas. Something is clearly wrong when VSRs are working long 
nights and weekends and still cannot meet their quotas. 

At first, management promised to revisit this problem but has since denied 
a problem even exists, claiming that the trend is improving even though there are 
still more than 20% of VSRs failing to meet the standards. This response is 
causing VSRs to leave or transfer. 

With no opportunity for collaboration, AFGE has had to invoke arbitration, 
which is currently pending along with a request for performance standard data to 
enable us to assess whether these standards are reasonable. A key issue raised 
by AFGE is that the current work credit (“weight”) system was developed before 
CPI specialization was put in place, and some employees are now exclusively 
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handling cases that receive less weight. We are asking VA management to work 
with us to reassess this process and design a system that has CPI in mind. The 
agreement that AFGE and VBA entered into last year also requires that 
management monitor the implementation of the national performance plan and 
make adjustments as needed. We encourage this Subcommittee to look into 
more effective ways to measure performance and ensure quality. . 
RECOMMENDATION: Implementation of the Skills Certification test should 
involve key stakeholders and include a clearly defined national training 
program designed to teach the subject areas and skills being tested. 

Pursuant to an agreement between VBA and AFGE, qualified GS-10 
VSRs who pass a skills certification test can receive a noncompetitive promotion 
to a GS-1 1 . AFGE has a number of concerns about the way this testing process 
has been implemented. 

First, contrary to assurances from VBA and the terms of our agreement, 
the training is not always sufficiently aligned with the scope of the exam, and 
trainers are often confused about what training materials are relevant to the test; 

There was significant variation in the amount of time employees had to 
train for the test. Even though the test is “open book", it tests for a tremendously 
high level of expertise. For example, an employee who got a wrong answer was 
directed to a 163- page user manual to find the right answer. More generally, it 
would be helpful to provide employees with test results and feedback using links 
to training syllabus, not source material. VSRs everywhere want to pass this test 
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and want the comprehensive and high quality training they deserve to 
accomplish this. 

To date, the tests resulted in extremely low passage rates (25% and 29% 
for the first two validity tests). This is very demoralizing to competent VSRs with 
years of experience, as well as an indicator of what a poor predictor the test is. 
Although the pass rate from the May 2006 test was somewhat higher (42%), we 
still find it very troubling that more than half the VSRs taking the test failed. 

We hope that other recent problems, such as inadequate notice of testing 
dates, duplicate test questions and trainer confusion, will be permanently 
resolved in the near future. 

AFGE was not allowed to collaborate in the process of refining the test 
and working out test administration problems. We were also denied access to 
raw test data to help address low passage rates. AFGE and veterans’ groups 
have a valuable role to play in improving the testing process if VBA permits it. 

The certification test raises the much larger issue of classification of the 
VSR and Rating Specialist positions. Currently, the VSR career ladder ends at a 
GS-10. The certification test is the only path to a GS-1 1 , Comparable 
adjudication jobs in other federal agencies have career ladders up to a GS-11. 
VBA has recently completed a review of the VBA claims adjudication 
classifications and submitted recommendations to the Secretary. Unfortunately, 
here too, AFGE was not allowed to have input into this classification review. 

We hope that this Subcommittee will look into the related questions of job 
classification and certification testing to determine the best approach to ensuring 
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that VSRs obtain the full set of skills they need to effectively serve veterans and 
that they are recognized for the skill levels they achieve. 

We look fonward to working with Chairman Miller and Ranking Member 
Berkley to ensure that meaningful training and performance standards are in 
place. To do otherwise would be a great disservice to veterans. Thank you. 
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ON 

THE TRAINING PROVIDED TO VETERANS BENEFITS ADMINISTRATION 
CLAIMS ADJUDICATORS AND THE STANDARDS USED TO MEASURE 
THEIR PROFICENCY AND PERFORMANCE 

SEPTEMBER 13. 2006 


Mr. Chairman and Members of the Subcommittee: 

Thank you for this opportunity to present The American Legion’s views on the training provided 
to Veterans Benefits Administration (VBA) claims adjudicators and the standards used to 
measure their proficiency and performance. We commend the Subcommittee for holding this 
hearing to discuss these important issues. 


TRAINING 


Proper mandatory training is a key factor in the quality of Department of Veterans Affairs (VA) 
regional office rating decisions. The Board of Veterans’ Appeals (BVA) combined remand and 
reversal rate (59.4 percent) of regional office decisions for Fiscal Year 2005 is a direct reflection 
of the lack of importance placed on training by the VA regional offices. Over the past few years. 
The American Legion’s Quality Review Team has visited almost 40 VA regional offices for the 
purpose of assessing overall operation. The American Legion reviews recently adjudicated 
claims and interviews service center staff Our site visits reveal that, at many regional offices, 
there have been too few experienced supervisors that could provide trainee adjudicators proper 
mentoring and quality assurance. In addition, at many stations, ongoing training for the new 
hires as well as the more experienced staff would be postponed or suspended, so as to focus 
maximum effort on production. 

Although the Under Secretary for Benefits has stated on numerous occasions that training of 
personnel is a top priority within VBA, the inconsistency in VBA’s training approach and in its 
implementation needs to be thoroughly reviewed and addressed by upper management within the 
Department. In the experience of The American Legion, the lack of proper training and oversight 
adversely impacts all areas of VBA. Please note that each of VBA’s 57 regional offices appear 
to have different approaches to training and also differ in the importance placed on training. 
According to a May 2005 report from the VA Office of the Inspector General (VAOIG) based 
on a survey of rating veteran service representatives (RVSRs) and decision review offices 
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(DROs), the respondents expressed positive opinions regarding the quality of their training but 
indicated that training has not received high priority. 

• 57 percent reported the quality of training to be good or very good 

• 1 6 percent reported the quality of training to be poor or very poor 

• 45 percent reported that they had received 10 hours or less of formal classroom 
instruction on rating policies and procedures in the last 12 months. 

• 24.1 percent reported that they had received 1 1-20 hours of formal classroom instruction 
in the last 12 months. 

• 18.0 percent responded that their regional office provides formal classroom instruction on 
rating policies and procedures once a week. 

• 45.6 percent responded that their regional office provides formal classroom instruction on 
rating policies and procedures once a month. 

• 36.4 percent responded that their regional office provides formal classroom instruction on 
rating policies and procedures once a quarter or less often. 

The information obtained in the VAOIG’s survey is consistent with what The American Legion 
has found in talking to service center staff during our quality review site visits. Some stations 
have regular formalized or structured training programs, while others have training programs that 
are best described as more informal and sporadic. Some stations have well established and 
structured training for new employees, but ongoing training for experienced staff is very limited. 

We are appreciative of the importance the Under Secretary for Benefits has placed on training of 
VBA personnel. We are also aware of the centralized training program that has been 
implemented; however, a national training standard/requirement, in addition to the centralized 
training conducted by Compensation and Pension Service (C&P), for regional office personnel is 
also needed. Consistent and standardized training at each regional office must take place for all 
personnel — experienced and new hires alike. The American Legion believes it is crucial that 
such a program be implemented and closely monitored for compliance by the Under Secretary 
for Benefits. Management in stations not in compliance with such training requirements must be 
held accountable; otherwise any national or centralized training effort will not be successful. 

Additionally, The American Legion believes it is essential to proper training that information 
(reasons for remand or reversal) from BVA decisions, DRO decisions and errors noted in 
National STAR and other internal quality reviews be tracked and examined for patterns. This 
information should then be used in mandatory formal training to ensure that common errors and 
other discrepancies occurring in regional office rating decisions are not repeated. This 
information should also be used for remedial training purposes when patterns of errors are 
identified for specific individuals. Although such data is currently being collected and 
disseminated to the regional offices, it appears that consistent utilization of this data in regular 
formalized and specific training has been lacking. Unless regional offices (both managers and 
individual adjudicators) leam from their mistakes and take corrective action, there will continue 
to be a high rate of improperly adjudicated claims, resulting in a consistently high appeals rate 
and subsequent high BVA remand/reversal rate of regional office decisions. 
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PERFORMANCE STANDARDS 


The emphasis on production continues to be a driving force in the VA regional office, often 
taking priority over such things as training and quality assurance. Performance standards of 
adjudicators and rating specialists are centered on productivity as measured by work credits, 
known as “End Products". Both veteran service representatives (VSRs) and rating veteran 
service representatives (RVSRs) have minimum national productivity requirements that must be 
met each day. Some stations also set their own standards, based on their claims backlog and 
other station specific requirements that is over and above the national requirement. 
Unfortunately, the end product work measurement system essentially pits the interests of the 
claimant against the needs of VA managers. The conflict is created because the regional offices 
have a vested interest in adjudicating as many claims as possible in the shortest amount of time. 
This creates a built-in incentive to take shortcuts so that the End Product can be taken. The 
system, in effect, rewards regional offices for the gross amount of work they report, not whether 
the work is done accurately or correctly, often resulting in many claims being prematurely 
adjudicated. These problems are caused by inadequate development, failure to routinely identify 
all relevant issues and claims and ratings based on inadequate examinations. Such errors are 
often overshadowed by the desire of VA managers to claim quick End Product credit. The result 
has been a traditionally high remand rate by the Board of Veterans’ Appeals (BVA) and the U.S. 
Court of Appeals for Veterans Claims (CAVC). The BVA’s combined remand and reversal rate 
(59.4 percent) for Fiscal Year 2005 is arguably a direct reflection of the greater emphasis placed 
on production over training and quality assurance. 

It seems to The American Legion that VBA management has been reluctant to establish a 
rigorous quality assurance program to avoid exposing the longstanding history of the 
manipulation of workload data and policies that contribute to poor quality decision-making and 
the high volume of appeals. VBA’s quality-related problems and the fact that little or no action 
is being taken to prevent or discourage the taking of premature End Products have been 
longstanding issues for The American Legion. The current work measurement system, and 
corresponding performance standards, are used to promote bureaucratic interests of regional 
office management and VBA rather than protecting and advancing the rights of veterans. The 
end product work measurement system, as managed by the VA, does not encourage regional 
office managers to ensure that adjudicators do the “right thing” for veterans the first time. For 
example, denying a claim three or four times in the course of a year before granting the benefit 
sought allows for a total of 5 end product work credits to be counted for this one case, rather than 
promptly granting the benefit and taking only one work credit. In the view of The American 
Legion, the need for a substantial change in VBA’s work measurement system is long overdue. 
A more accurate reliable work measurement system would help to ensure better service to 
veterans. Ultimately, this would require the establishment of a work measurement system that 
does not allow work credit to be taken until the decision in the claim becomes final, meaning that 
no further action is permitted by statute whether because the claimant has failed to initiate a 
timely appeal or because the BVA rendered a final decision. 
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PROFICIENCY/COMPETENCY 


C&P conducted an open book (pilot) job skill certification test for VSRs several years ago in 
which the pass rate was extremely low (approximately 23 percent). Even more alarming than the 
low-test scores was the fact that those who took the test had several years of experience in the 
position and were considered to be proficient. 

C&P subsequently finalized its VSR proficiency test and conducted tests in May and August of 
this year. Employees participating in the testing underwent 20 hours of training prior to taking 
the test and the success rate (approximately 42 percent) for the May test was much higher than 
the pilot test. The results for the August test have not been released yet. C&P plans on 
conducting two VSR tests each year, one in winter and the other in the summer. 

The American Legion applauds the new testing program as a step in the right direction but we 
still have concerns. Although successful completion of the test will be required for promotion or 
assignment to a rating board, it is not mandatory as a condition of employment in that position. 
C&P is in the process of developing a test for RVSRs and DROs but a timeline for completion or 
implementation has not yet been determined. Unfortunately, like the VSR test, the test for 
RVSRs and DROs will not be mandatory as a condition of employment. 

The goat of competency testing is to ensure that an individual in any given position is 
competent, proficient, and otherwise qualified to perform the duties required of that position. 
This testing goal will not be achieved if testing is not mandatory or is not provided for all levels 
or for all positions. 


CLOSING 


The American Legion realizes that VBA faces many difficult challenges during the upcoming 
fiscal year. Although we have offered our suggestions and comments, we realize that there is no 
easy solution and we will continue to work closely with VA to ensure our nation’s veterans 
receive the benefits to which they are entitled. That concludes my testimony. I will be happy to 
answer any questions. 
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